Spherical forums

Sphere Development => Engine Development => Topic started by: Fat Cerberus on January 25, 2015, 11:21:21 am

Title: neoSphere 5.9.2
Post by: Fat Cerberus on January 25, 2015, 11:21:21 am
This is neoSphere, the current official implementation of Sphere and a complete rewrite of Chad Austin's original Sphere game engine, coded from the ground up in C.  It uses Allegro (http://alleg.sourceforge.net/) for cross-platform graphics and sound and JavaScript is powered by ChakraCore (https://github.com/Microsoft/ChakraCore).  The engine boasts near-full parity with the Sphere 1.5 API and most Sphere 1.x games will run in neoSphere with no modifications.  But make no mistake: this is a Sphere v2 engine, and includes many, many features not found in the engine it replaces.  These include Galileo, a scene graph API pioneered by TurboSphere which enables more flexible rendering than the Sphere v1 primitive-drawing functions; the RNG class for ultimate control over random number generation; SphereFS, a standard protocol for specifying asset filenames within the sandboxed file system; and full native support for JavaScript module syntax (import/export) as defined in ES2015.  Best of all, old and new API functions can be mixed within the same codebase, making migration of existing codebases to Sphere v2 very easy.

neoSphere natively supports many modern JavaScript features such as template strings, arrow functions, destructuring assignment, even modules!  The engine also comes with an SCons-inspired programmable build engine, Cell, also JS-powered, which can be used to build assets on-the-fly.  neoSphere was also the first implementation of Sphere to include support for single-step debugging!  Thanks to ChakraCore's powerful built-in debugger API, it does!  Using either Sphere Studio or the SSj command-line debugger (both included with the engine), you can set breakpoints, see console output, and step through your game's code line-by-line to find bugs, inspect variables and even run arbitrary JavaScript code.

neoSphere 5.9.2 - December 22, 2023 - Release Notes (https://github.com/spheredev/neosphere/releases) - GitHub Repo (https://github.com/spheredev/neosphere)
Includes neoSphere, SpheRun, Cell compiler, and SSj command-line debugger.



Sphere v2 API Documentation
Title: Re: minisphere
Post by: Fat Cerberus on January 25, 2015, 07:26:51 pm
That's odd, I just got an email that Flying Jester replied to the topic, but there's nothing here...
Title: Re: minisphere
Post by: Flying Jester on January 25, 2015, 07:42:13 pm
I almost wrote a reply, but then I decided to investigate first.

SFML ;_;
I never tried to use it before, but it's a bit of a mess on OS X. I brought the engine up on OS X (mostly) using SCons and the prebuilt SFML libraries for OS X.
Title: Re: minisphere
Post by: Fat Cerberus on January 26, 2015, 09:58:43 pm
How up-to-date is the api.txt that comes with Sphere?  I'll need a reference in order to replicate the Sphere 1.x API.
Title: Re: minisphere
Post by: Flying Jester on January 26, 2015, 11:41:33 pm
Last I checked, it has a few functions that don't actually work in Sphere (surface.blitImage() jumps to mind). I haven't found anything it's missing yet

I've been using it (from my git repo) to fill out the TS map engine.

I'm curious where the line will be drawn between script and native. Are surfaces native or script?
Title: Re: minisphere
Post by: Fat Cerberus on January 27, 2015, 12:10:24 am
How would I do surfaces in script?  That definitely seems like something that should be native.
Title: Re: minisphere
Post by: Flying Jester on January 27, 2015, 03:40:02 am
It would be simple, just an object that has a UInt32Array, a width, and a height :)
Title: Re: minisphere
Post by: Fat Cerberus on January 27, 2015, 12:43:36 pm
Small issue with that: besides that that puts surface operations entirely in software, how would I implement methods like surface.drawText, not to mention all the primitives.  If I wanted a 1:1 mapping to Sphere 1.x I'd have to implement those all in JS as well, meaning I don't get the benefit of letting the hardware do it, as I could if I were to implement surfaces natively.
Title: Re: minisphere
Post by: Fat Cerberus on January 28, 2015, 10:51:29 pm
So I'm thinking maybe not surfaces, but a lot of things will be implemented scriptside in minisphere.  For example: Color objects.  Duktape's stack-based API is painful to use (I don't even remember Lua's being this bad...) and once constructors and prototypes get involved, it gets even more convoluted.  For simple objects like Color, there's no real benefit to creating them on the C side as in the end it's just a simple RGBA tuple.  JS can handle those very well on its own.

In the process I think I'm going to make tweaks to the structure of the API, clean things up a bit.  For example, BlendColors() and BlendColorsWeighted() both get folded into the Color object as a single method: color_obj.blendWith(color2, w1, w2).  Of course, to maintain compatibility with existing Sphere code, I'll be sure to expose some kind of legacy API, likely as an optional thing, similar to what Jester is planning to do with TurboSphere.
Title: Re: minisphere
Post by: Flying Jester on January 29, 2015, 02:28:31 am
I've been doing that to a certain extent with the map engine, and with images now.

I think that colors are one of the better objects to do that with, actually. In V8 there was a pretty high overhead to making any native-defined object, especially since that usually meant some native-side allocation had to occur. Making colors is easy to do all over the place in a normal Sphere script without even thinking about it. Putting them fully in script would help alleviate that. Especially because there is no particular reason they need to be done in native.

I think it's a good technique. Have a powerful, concise core API that Sphere's API is implementable in, and also a script-side layer that fills out the rest of the Sphere API. Even with the map engine, which is all script anyway =P
Title: Re: minisphere
Post by: Fat Cerberus on January 29, 2015, 10:32:23 pm
I'm not too concerned about script-to-native overhead--the purpose of a stack-based API like the one in Duktape and Lua is that any objects created are allocated in-engine, rather than on the native side.  For example, this:

Code: (c) [Select]
duk_push_global_object(ctx);
duk_push_object(ctx);
duk_push_int(ctx, 812);
duk_put_prop_string(ctx, -2, "number");
duk_put_prop_string(ctx, -2, "myObj");
duk_pop(ctx);


...is quite literally equivalent to this:
Code: (c) [Select]
duk_eval_string_noresult(ctx, "myObj = {}; myObj.number = 812;");


As you can probably tell, the bigger concern here is the unmaintainable mess the C-side constructors will end up becoming (hint: see first code snippet).  A lot of that can be avoided simply by implementing stuff directly in JS to start with.
Title: Re: minisphere
Post by: Fat Cerberus on January 30, 2015, 10:45:49 pm
Hm, so I'm thinking I may try out Allegro after all.  SFML was definitely designed for use in C++ (mapping a C++ API directly to C = incredibly bad idea), and I'd really like to keep this in C to keep it as simple as possible.
Title: Re: minisphere
Post by: Flying Jester on January 31, 2015, 12:35:33 am
I support this idea!

SFML is also not very friendly to OS X or other more alternative OS's (neither is CMake, which is primarily what stopped me from manhandling it).

Have you considered SDL2 (or SDL 1.2 if you want super-mega portability)? It's a very cleanly C library.
Title: Re: minisphere
Post by: Fat Cerberus on January 31, 2015, 12:18:26 pm
I hadn't thought about SDL.  I'll consider it, especially after seeing how much stuff allegro has with all those plugins.  How much portability would I be losing to go with SDL2 over 1.2?
Title: Re: minisphere
Post by: N E O on January 31, 2015, 01:05:21 pm

How much portability would I be losing to go with SDL2 over 1.2?


The main difference between SDL2 and 1.2 is API changes, especially WRT window handling (which is one of the many things abstracted to their newer image/surface/frame/etc buffer handling API). Most of the extra libraries (especially sdl_audio) do keep their APIs but simply changed internally to be SDL2-compatible at that time.

Support-wise, whatever 1.2 runs on SDL2 still runs on, but SDL2 might recommend a more C++ approach to it.

Re Allegro - It won't hurt to see how far an Allegro-based Sphere-compatible engine can go. If you choose to do it, I'd very much consider promoting it as an official multi-platform alternative to current vanilla 1.5, alongside TurboSphere and SSFML. :)
Title: Re: minisphere
Post by: Flying Jester on January 31, 2015, 09:38:14 pm
NEO: That's not entirely true. SDL2 does not work on some platforms like QNX, SunOS, Windows CE, BeOS, and AmigaOS4 where SDL 1.2 did work, and support is spotty on Solaris, AIX, HP-UX, and Haiku.

So basically, you lose some old-world mobile platforms and QNX, and it becomes hard going on traditional, commercial Unix, and Haiku. I for one do not mourn for the platforms that become unusable. Having Unix, Windows (Win32 and Metro), Haiku, Android, and iOS, in addition to a theoretical HTML5/JSM platform seems like plenty to me.

SDL2 supports compiling out certain portions, too (for instance, the SDL that I distribute with TS has no audio support), but I found that SFML can do that too, and I suspect Allegro could as well.

SDL2 (and for the most part SDL 1.2) are a mix of OOP and procedural code. I find it pretty easy to work with in either C or C++, it doesn't feel all that unnatural in either. I'd say its worth considering, Allegro is fine with me, too. These are all fairly comparable solutions, and I wouldn't say that its 'wrong' to choose any of them.
Title: Re: minisphere
Post by: Fat Cerberus on February 01, 2015, 02:37:57 am
So Allegro's API is actually quite nice.  Much less boilerplate required to get things up and running compared to SFML.  Just create a display and an event queue, and then a main loop that calls al_flip_display() and processes events and you're good to go.  SFML was such a pain in the ass, requiring creating structures all over the place just to do something as simple as draw a rectangle.

Honestly, Allegro seems like a perfect fit for the backend of a Sphere engine: Its API is very close in structure to the way the Sphere 1.x API is already designed, meaning this should be pretty painless.
Title: Re: minisphere
Post by: Radnen on February 01, 2015, 01:11:10 pm

So Allegro's API is actually quite nice.  Much less boilerplate required to get things up and running compared to SFML.  Just create a display and an event queue, and then a main loop that calls al_flip_display() and processes events and you're good to go.  SFML was such a pain in the ass, requiring creating structures all over the place just to do something as simple as draw a rectangle.


I found the C# binding far easier to use.


Honestly, Allegro seems like a perfect fit for the backend of a Sphere engine: Its API is very close in structure to the way the Sphere 1.x API is already designed, meaning this should be pretty painless.


I agree.
Title: Re: minisphere
Post by: Fat Cerberus on February 01, 2015, 11:31:40 pm
What should I return from GetVersion()?  I'm assuming 1.5 since that's the API I'm shooting to be compatible with...
Title: Re: minisphere
Post by: Radnen on February 01, 2015, 11:40:51 pm

What should I return from GetVersion()?  I'm assuming 1.5 since that's the API I'm shooting to be compatible with...


Yes, otherwise some games won't work. ;)
Title: Re: minisphere
Post by: Fat Cerberus on February 02, 2015, 02:07:47 am
So fairly big showstopper I just hit: Duktape creates and writes JS objects entirely in-engine with the only thing existing on the native side being the code of native functions.  There doesn't appear to be a way to create an object on the native side (e.g. a sound) and have data associated with it that is invisible to JS (pointers or handles, say) but readily accessible to native code.  The only facility Duktape supplies for this purpose is the "stash", which is itself an in-engine (but invisible to script) JS object and global to the JS context--not exactly suitable.

So yeah, not really sure what to do here.

edit: Also, Allegro's handling of alpha is... odd.  I draw two overlapping rectangles, both with alpha 0.  These should be completely invisible, right?  Instead I get this...

edit 2: Well, I figured it out the alpha issue anyway, Allegro's default blend mode assumes premultiplied alpha, so I just had to change it using al_set_blender()
Title: Re: minisphere
Post by: Fat Cerberus on February 04, 2015, 10:59:15 pm
Okay, so Duktape is very strange.  If you try to retrieve the location (file and line number) of a script error, it instead gives you the filename of the C source file which called the script and the line in that file where the Duktape call was made.  For instance, calling Abort() reports that the error happened in sphere_api.c on line 49--in other words, the duk_error call.  Even for, say, syntax errors though, it seems to correctly report the line in the script, but not the filename--instead it returns the path of the C file making the duk_peval_*() call.

Here's where things get strange though: It correctly reports the source file and line number for the duk_error() call even in release mode, where that information shouldn't even be available (there's no debugging info).  What sorcery is this?
Title: Re: minisphere
Post by: Flying Jester on February 05, 2015, 03:12:09 am
I'll guess that it's a macro, and uses the __LINE__ and __FILE__ macros. That's how assert.h's assert() does it.
Title: Re: minisphere
Post by: Fat Cerberus on February 06, 2015, 03:00:41 pm
Yeah, that's what it is, half of Duktape's API is just macros that call internal functions.  Thank goodness Intellisense works with macros now (it never used to, at least it didn't the last time I used VC++), otherwise that would be annoying as hell.

I also did some reading and found out I can store internal state for native-created JS objects.  It's simple, too: On the native side, you just set properties on the object with identifiers starting with 0xFF as the first character.  Since that creates an invalid UTF-8 string, such properties aren't accessible on the JS side but the native code can access them easily.
Title: Re: minisphere
Post by: Flying Jester on February 06, 2015, 06:49:35 pm
Does that mean that duktape makes no distinction from numerically-defined properties and string-defined properties?

I actually don't if a JS engine properly has to, but both V8 and SM make a clear distinction there.
Title: Re: minisphere
Post by: Fat Cerberus on February 06, 2015, 07:43:04 pm
Technically, at least according to the Duktape docs, the JS spec says that all properties (including array indices!) are canonically keyed by string, however:

http://duktape.org/guide.html#programming.6

Duktape special-cases arrays so that if you're only accessing properties by number, it's fast.
Title: Re: minisphere
Post by: Fat Cerberus on February 07, 2015, 11:44:50 am
So something interesting I found out: Duktape uses reference counting for garbage collection, only falling back on mark/sweep if refcounting fails to free the object--i.e. circular refs.  Which means a call like the following:

Code: (javascript) [Select]
LoadSound('bgm.ogg').play();


...will cause the garbage collector to finalize and free the sound as soon as play() returns (meaning this snippet is effectively an expensive no-op under Duktape), whereas, say, SpiderMonkey, the sound might actually stick around for a while.  Something to keep in mind.

As for actual minisphere news, I've implemented about half of the Sound API (NOT the SoundEffect API though, that's a different beast entirely), meaning minisphere can load and play music and sounds now!  It definitely helps that the learning curve on Allegro (and even Duktape, for the most part) is practically non-existent, which is awesome.  Now I remember why I used to love coding in C.  Sure, it's ridiculously easy to shoot yourself in the foot (although perhaps not quite so bad as C++), but the language is so simple that, for the most part, it ends up being self-documenting.
Title: Re: minisphere
Post by: DaVince on February 07, 2015, 03:56:26 pm
Nice work on this! I've quietly been following it in the background.


Yeah, that's what it is, half of Duktape's API is just macros that call internal functions.

If that's how it works, the name "Duktape" sounds very appropriate.

Quote
NOT the SoundEffect API though, that's a different beast entirely

Well, SoundEffect wasn't a part of Sphere 1.5 anyway. :)
Title: Re: minisphere
Post by: Fat Cerberus on February 07, 2015, 04:03:56 pm


Yeah, that's what it is, half of Duktape's API is just macros that call internal functions.

If that's how it works, the name "Duktape" sounds very appropriate.

Quote
NOT the SoundEffect API though, that's a different beast entirely

Well, SoundEffect wasn't a part of Sphere 1.5 anyway. :)


The api.txt that comes with Sphere 1.5 includes it... nothing like documenting a feature that doesn't exist.
Title: Re: minisphere
Post by: DaVince on February 07, 2015, 04:04:45 pm
Oh, that's interesting. I was pretty sure FBNil added the feature in 1.6... but I guess I was wrong then? :P
Title: Re: minisphere
Post by: Radnen on February 07, 2015, 04:19:54 pm

Oh, that's interesting. I was pretty sure FBNil added the feature in 1.6... but I guess I was wrong then? :P


It was experimental, I think. FBNil made it non-experimental. But then 1.6 is still an alpha version of Sphere still to this day.
Title: Re: minisphere
Post by: Fat Cerberus on February 07, 2015, 05:06:42 pm
So is there a safer (not subject to buffer overflows) version of sprintf() I can use?  sprintf_s() is VC++-only, and C99 support in VC is abysmal, meaning no snprintf() either.  See, my code for normalizing paths looks like this:

Code: (c) [Select]
char*
normalize_path(const char* path, const char* base_dir, char* out_abs_path)
{
char* game_path = "C:/cowz";
char* norm_path = strdup(path);
size_t path_len = strlen(norm_path);
for (char* c = norm_path; *c != '\0'; ++c) {
if (*c == '\\') *c = '/';
}
if (norm_path[0] == '/' || norm_path[1] == ':')
// absolute path - not allowed
return NULL;
bool is_homed = (strstr(norm_path, "~/") == norm_path);
if (is_homed) {
sprintf(out_abs_path, "%s/%s", game_path, norm_path + 2);
}
else {
sprintf(out_abs_path, "%s/%s/%s", game_path, base_dir, norm_path);
}
free(norm_path);
return out_abs_path;
}


Which is obviously subject to buffer overflow.  Not a good thing considering Sphere takes JS as input--too much leeway for exploits using overly-long paths.
Title: Re: minisphere
Post by: Flying Jester on February 07, 2015, 08:38:42 pm
You should check out using parts of BSD's libc. That's what I've been doing for my Kashyyk IRC client when I want some reasonable C-string support with MSVC.

For instance, I imported BSD libc's strcasestr function (look here, strcasestr.c, strcasestr.h, str-two-way.h (https://github.com/FlyingJester/Kashyyyk/tree/master/kashyyyk/platform)) for use with MSVC (and in more recent builds, MingW as well). I only compile the functions I need, not the entire library.
BSD's libc seems to be intended to work with pretty archaic and incapable compilers, and so it seems to mostly work with even MSVC, which it has no real reason to support. It also has very confined systems, so in the case of strcasestr, I only needed one extra file besides the source file and header for the function I wanted (and even then just a header).

Alternatively, just put in some defines or def'ed out functions that adapt snprintf, sprintf+strlen, and sprintf_s depending on what is available to the compiler. That way you can get sprintf_s and snprintf when they are available, and have a safe, if less efficient default when neither exists. I do that for `strdup' in TurboSphere. One define that uses strdup, one with strdup_, an inline function that uses strlen, malloc, and memcpy to replicate it.
Title: Re: minisphere
Post by: Radnen on February 07, 2015, 09:10:05 pm

Alternatively, just put in some defines or def'ed out functions that adapt snprintf, sprintf+strlen, and sprintf_s depending on what is available to the compiler. That way you can get sprintf_s and snprintf when they are available, and have a safe, if less efficient default when neither exists. I do that for `strdup' in TurboSphere. One define that uses strdup, one with strdup_, an inline function that uses strlen, malloc, and memcpy to replicate it.


This.
Title: Re: minisphere
Post by: Fat Cerberus on February 08, 2015, 01:59:14 am
...and I've just hit the one thing I absolutely hate about C: String processing.  Even just to parse the command line to interpret -game "<game dir>" is ridiculous.  Also, Duktape isn't particularly fast--in fact, I think it might even be slower than the SM in Sphere 1.5.  I had minisphere try to run Specs (which has lots of RequireScript in main.js), and there was about a half-second delay between the game window showing up and the JS error due to undefined identifiers (i.e. unimplemented APIs).  Granted, this was under the VC debugger, so I'm sure I took a performance hit just for that, but even so...

I'm starting to wonder how viable this project actually is. :(
Title: Re: minisphere
Post by: Flying Jester on February 08, 2015, 02:29:48 am
You could check out using stdargs (or was it stdarg?) for that. Or for more generally strings, something the NSPR. There is a reason that a LOT of C-based projects either have or use some system other than just raw C-strings for their string representations.

It shoudn't be that hard to do arg parsing like that in C. TurboSphere did its argument parsing in pure C up until very recently.

It's possible that duktape is much slower on error conditions (like an uncaught exception). That's true of a lot of virtual machines, especially because often there's not a lot of reason to be fast when you can't recover.
Title: Re: minisphere
Post by: Fat Cerberus on February 08, 2015, 10:47:46 am
Yeah, I found out from playing around that the game dir comes into main() as argv[1] = "-game" and argv[2] = <game dir> conveniently sans quotes, making it simpler than expected.  Of course, this was AFTER writing code to parse the full command line... oh well.

Can I count on this behavior though, or do there exist compilers that pass the full command line to main() as a single argument?

Also, MS apparently doesn't like strdup and getcwd (and probably some others).  The compiler complains that "The POSIX name is deprecated, use _strdup instead."  Last I checked, these weren't deprecated in any standard...  I would really like a clean build without preprocessor cruft everywhere, but that's looking more and more impossible.  Pure C is apparently not as portable as everyone makes it out to be.
Title: Re: minisphere
Post by: Flying Jester on February 08, 2015, 02:09:41 pm
You can count on un-escape spaces to separate arguments.

Code: [Select]

./engine -game startup
# argv[0] engine
# argv[1] -game
# argv[2] startup


Code: [Select]

./engine -game\ startup
# argv[0] engine
# argv[1] -game startup


Code: [Select]

./engine "-game startup"
# argv[0] engine
# argv[1] -game startup


This is true on bash and sh compatible shells, and on CMD. So you can properly count on it.

The posix names are deprecated in MSVC. `strdup' is, in fact, not pure C. If you want to be portable, you should consider something like this:

Code: [Select]


#ifdef _MSC_VER
#define STRDUP strdup_
#else
#define STRDUP strdup
#endif



Recently I've not been doing that, though. I begin to seriously doubt that MSVC's standard library will ever lose the posix names.

You can see on github how I dealt with paths in Kashyyyk, which is what how TurboSphere does it is based on:

https://github.com/FlyingJester/Kashyyyk/blob/master/kashyyyk/platform/win32_paths.c (https://github.com/FlyingJester/Kashyyyk/blob/master/kashyyyk/platform/win32_paths.c)
https://github.com/FlyingJester/Kashyyyk/blob/master/kashyyyk/platform/unix_paths.c (https://github.com/FlyingJester/Kashyyyk/blob/master/kashyyyk/platform/unix_paths.c)
https://github.com/FlyingJester/Kashyyyk/blob/master/kashyyyk/platform/cocoa_paths.m (https://github.com/FlyingJester/Kashyyyk/blob/master/kashyyyk/platform/cocoa_paths.m)
Title: Re: minisphere
Post by: Fat Cerberus on February 10, 2015, 07:23:15 pm
Let me just say for the record that Allegro is awesome.  So much stuff is built in that I never expected to be, I always thought it was just a graphics library.  For instance, a platform-agnostic way of handling file paths, avoiding all the path-parsing headaches discussed above.  Also: INI-style config management, meaning I don't even have to parse game.sgm myself, I can let Allegro do it. ;D

One potential source of incompatibility, though: Duktape doesn't recognize const, it causes a syntax error.  I realize it's not standard JS, but SpiderMonkey handles it without issue, so any existing Sphere game that declares things const won't run in minisphere.
Title: Re: minisphere
Post by: Radnen on February 10, 2015, 09:05:11 pm

One potential source of incompatibility, though: Duktape doesn't recognize const, it causes a syntax error.  I realize it's not standard JS, but SpiderMonkey handles it without issue, so any existing Sphere game that declares things const won't run in minisphere.


Same with Jurassic! I was able to modify the code though so I am using technically a modified version of it. You could do a RegEx replacement of const to var, but you have to make sure it's only ever in the context of a variable declaration and not say, in some text or a comment (meaning you'll have to write a custom pre-processor parser).
Title: Re: minisphere
Post by: Fat Cerberus on February 11, 2015, 10:42:50 am
Okay, so I think minisphere actually isn't too far off from being able to run Spectacles.  I think I just need surface support, which shouldn't be too difficult, and .rfn font support (plus associated text-drawing functions), which... may be harder.

See, Allegro has a function to register your own font loader, al_register_font_loader(), which I could use to let it load .rfn fonts, but here's the issue: The documentation for the ALLEGRO_FONT structure is, well, pretty much nonexistent--even a Google search turns up nothing of use.  Which leaves me only to look at how Allegro loads fonts internally to try to figure it out myself, and even that wasn't particularly helpful at last attempt.

I knew I had to hit a snag eventually.
Title: Re: minisphere
Post by: Flying Jester on February 11, 2015, 04:52:16 pm
It's not too difficult to even draw bitmap fonts yourself. Especially if you are using an immediate API.

I actually think .rfn (and .rws) support would best be done in script. I'm going to do that with TurboSphere.
Title: Re: minisphere
Post by: Fat Cerberus on February 11, 2015, 05:43:34 pm
See, I would do stuff in script, but Duktape is actually pretty slow so I need all the performance help I can get by doing stuff natively.  Besides, if I let allegro load the font instead of drawing the bitmaps manually, I get stuff like WrapText etc. basically for free instead of having to implement it myself.
Title: Re: minisphere
Post by: Fat Cerberus on February 12, 2015, 02:16:28 pm
So I finally got Allegro loading RFN fonts, and it actually seems to work pretty well.  Only issue: Apparently RFN fonts are actually stored fixed-width (including the default system font), despite vanilla Sphere rendering them variable-width--and the format supports it even!  This means naively rendering characters according to their associated glyphs' widths causes the string to be spaced quite far apart.  Still haven't figured out how to fix this yet.

I guess I could crop the glyph bitmaps at load time, but I'm reluctant to do so because that would mess with RFNs that were actually saved variable-width and would thus already be tuned for proper kerning, etc.

EDIT: Nevermind, ignore me, I'm an idiot.  The cause of my incorrect text rendering was this brain fart here:
Code: (c) [Select]
static int
rfn_render_char(const ALLEGRO_FONT* f, ALLEGRO_COLOR color, int ch, float x, float y)
{
rfn_font_t* rfn = f->data;
rfn_glyph_t* glyph = &rfn->glyphs[ch];
al_draw_tinted_bitmap(glyph->bitmap, color, x, y, 0x0);
return glyph->header.height; // <-- see, it's supposed to return the WIDTH of the char...
}
Title: Re: minisphere
Post by: N E O on February 13, 2015, 04:57:01 pm
Just out of curiosity, is anyone else here seeing centered code blocks on Lord English's C examples? I want to double-check if I need to edit more than one theme to fix that.
Title: Re: minisphere
Post by: DaVince on February 13, 2015, 05:19:37 pm
Looks like this here: http://i.imgur.com/C6o9Lgk.png
Title: Re: minisphere
Post by: Flying Jester on February 13, 2015, 05:40:51 pm
@NEO They appear correctly for me in Fx. I tested in Safari and Lynx too, just because you mentioned it, no issues there either.
Title: Re: minisphere
Post by: Radnen on February 13, 2015, 09:26:21 pm
NEO: Seems good on the default theme.
Title: Re: minisphere
Post by: N E O on February 14, 2015, 12:54:35 pm
Ok, so the issue is only with my unreleased Spherical theme; I know what needs to be fixed and how, I just wanted to make sure it was only one theme.
Title: Re: minisphere
Post by: DaVince on February 14, 2015, 02:47:59 pm
I thought I was supposed to have the Spherical theme. Could you enable it for me?
Title: Re: minisphere
Post by: Fat Cerberus on February 15, 2015, 10:00:37 am
Spectacles runs in minisphere!  There seems to be a good deal of lag on slower machines, though.  Not sure yet if the bottleneck is rendering-related (Allegro) or Duktape though.
Title: Re: minisphere
Post by: Radnen on February 15, 2015, 11:31:31 am
Wow that was fast!
Title: Re: minisphere
Post by: Fat Cerberus on February 15, 2015, 11:58:49 am
Yeah, as it turns out the Sphere 1.5 API isn't all that complicated, there's just a LOT of functions.  Most of the graphics APIs would be about 3-5 lines of code if it weren't for all the stack juggling Duktape requires.

So, who's up for trying to compile minisphere on something other than Windows to see how it works? :D
Title: Re: minisphere
Post by: N E O on February 15, 2015, 02:29:37 pm

I thought I was supposed to have the Spherical theme. Could you enable it for me?


It says you already have it, hm...
Title: Re: minisphere
Post by: Flying Jester on February 15, 2015, 10:23:07 pm

Yeah, as it turns out the Sphere 1.5 API isn't all that complicated, there's just a LOT of functions.  Most of the graphics APIs would be about 3-5 lines of code if it weren't for all the stack juggling Duktape requires.


Implementing the Sphere graphics API was precisely what lead me to believe that it would be better to have a very small API, even if it were a little more verbose. That's why TurboSphere just has the Shape and Group objects, and no primitives or even direct Surface or Image blitting at all.


So, who's up for trying to compile minisphere on something other than Windows to see how it works? :D


I can give it a try on some other platforms.

I already tried earlier, and I found the following:

Solaris US: Doesn't work, Allegro doesn't work there.
FreeBSD: Same
Haiku: Same

But I assume you are more interested in Linux and OS X.
Title: Re: minisphere
Post by: Fat Cerberus on February 15, 2015, 11:23:34 pm


Yeah, as it turns out the Sphere 1.5 API isn't all that complicated, there's just a LOT of functions.  Most of the graphics APIs would be about 3-5 lines of code if it weren't for all the stack juggling Duktape requires.


Implementing the Sphere graphics API was precisely what lead me to believe that it would be better to have a very small API, even if it were a little more verbose. That's why TurboSphere just has the Shape and Group objects, and no primitives or even direct Surface or Image blitting at all.


So, who's up for trying to compile minisphere on something other than Windows to see how it works? :D


I can give it a try on some other platforms.

I already tried earlier, and I found the following:

Solaris US: Doesn't work, Allegro doesn't work there.
FreeBSD: Same
Haiku: Same

But I assume you are more interested in Linux and OS X.


Yeah, Linux and OS X are my main concerns.  Honestly I don't know anyone that uses Solaris or FreeBSD (although I assume the latter is more common), and I've never even heard of Haiku.
Title: Re: minisphere
Post by: Radnen on February 16, 2015, 12:32:26 am
I keep getting a "syntax error: parse error (line 10)". I've tried to run the startup game or other games, but with no luck.

I'm also surprised that for a minisphere it's the same size on disk as my SSFML. But the memory use should be much, much shallower in yours.
Title: Re: minisphere
Post by: Fat Cerberus on February 16, 2015, 01:03:54 am

I keep getting a "syntax error: parse error (line 10)". I've tried to run the startup game or other games, but with no luck.

I'm also surprised that for a minisphere it's the same size on disk as my SSFML. But the memory use should be much, much shallower in yours.


Parse error means it encountered const.  If you change the consts to vars, you'll have a better shot at running it.

Also: It's still smaller than Sphere 1.5.  About half the size at last check.  Also a much smaller codebase.  Vanilla Sphere's codebase is frankly huge for such a simple engine.
Title: Re: minisphere
Post by: Flying Jester on February 16, 2015, 01:18:59 am

Also: It's still smaller than Sphere 1.5.  About half the size at last check.  Also a much smaller codebase.  Vanilla Sphere's codebase is frankly huge for such a simple engine.


Even if you include all the Allegro libraries? Sphere's codebase includes the entire editor, too (which has some guts in many shared components), as well as the loaders for many audio file types, image types (including all of libungif), and the binaries include the MSVC runtime, as well as several graphics plugins.

I'd be surprised if, including all the necessary libraries, minisphere was actually able to open all the same file types as Sphere, play most of the same games (stuff like const notwithstanding), and was still less than half the size.
Title: Re: minisphere
Post by: Radnen on February 16, 2015, 01:25:57 am
It still says that error no matter how simple of a script I put in it, even if it has <10 lines code.
Title: Re: minisphere
Post by: Fat Cerberus on February 16, 2015, 01:29:51 am
@FJ:
I'm using the Allegro monolithic library, which is about a 2.25MB DLL and includes all Allegro functionality, including audio (if you're using more than about 3 Allegro components, the monolithic is actually smaller than the individual DLLs).  The minisphere executable at current is around 550KB.  That's literally the entire engine in 2.8MB.  Even if you count the system directory (which I guess technically you should? Full Sphere 1.5 compliance and all): 3.89MB.

Admittedly not a huge difference--ignoring editor.exe and SciLexer.dll, but including config.exe and the system directory, Sphere 1.5 is 5.25MB.  Still, minisphere is smaller, so I think the name still stands! :)

@Radnen:
Very odd.  Give me a code sample that you tried that caused the error, I'll look into it.  It runs Spectacles very well, so...
Title: Re: minisphere
Post by: Radnen on February 16, 2015, 01:32:44 am

@Radnen:
Very odd.  Give me a code sample that you tried that caused the error, I'll look into it.  It runs Spectacles very well, so...


Well, it does it for anything, even if I use dll files, etc. how are you playing games with it? Do you drag them on to the executable or do you put everything into the startup folder? Because the startup game also did the same thing.
Title: Re: minisphere
Post by: Fat Cerberus on February 16, 2015, 01:34:16 am
You can put the game into the startup folder, but actually, all you have to do is point Sphere Studio at minisphere and it'll work.  It accepts the -game commandline argument just like 1.5 and SSFML does.
Title: Re: minisphere
Post by: Fat Cerberus on February 17, 2015, 03:13:39 am
So did you get it to run anything?  Specs uses a good chunk of the API, so I'd expect there must be at least some games out there that work.  Although most games use the map engine, so perhaps not...  Note that the startup game won't work even if you remove the consts as I didn't implement ExecuteGame et al. yet.

Anyway, I've just started on the map engine and am currently trying to get In the Steps of the Blackfoot running.  That was the game that introduced me to Sphere, so I kind of have a soft spot for it. :)
Title: Re: minisphere
Post by: Flying Jester on February 17, 2015, 03:19:28 am

Note that the startup game won't work even if you remove the consts as I didn't implement ExecuteGame et al. yet.


ExecuteGame is very challenging to implement with most JS engines. It's extremely difficult, or just plain impossible for V8 to cancel execution of a script from native. I am unsure of how difficult it is with SM, although it seems to have better support for such an idea. I would think that JS engines not expressly intended for a web browser may actually have an advantage here.

I basically have just written this part of the API off. It's a cool feature, but ultimately not that useful. I don't lose much sleep knowing the startup games don't run.
Title: Re: minisphere
Post by: Fat Cerberus on February 17, 2015, 03:27:11 am
That's the awesome thing about Duktape--everything is done in-engine, and the call to game() is protected, meaning all I have to do to bail out of script execution (including infinite while(true) loops!  Assuming they call FlipScreen anyway...) is throw an exception via duk_error().  This drops me off immediately after the duk_pcall, with the exception on the stack.  This is the same mechanism I use to bail out when the game window is closed, and it's awesome.
Title: Re: minisphere
Post by: Radnen on February 17, 2015, 12:32:44 pm

I basically have just written this part of the API off. It's a cool feature, but ultimately not that useful. I don't lose much sleep knowing the startup games don't run.


Not so fast! There is a... compromise. You could spawn an entirely new instance of Sphere. So you would have two games running simultaneously, but if you minimize one and play the other you should be fine. I'm intending to do this with SSFML.
Title: Re: minisphere
Post by: Flying Jester on February 17, 2015, 05:16:36 pm


I basically have just written this part of the API off. It's a cool feature, but ultimately not that useful. I don't lose much sleep knowing the startup games don't run.


Not so fast! There is a... compromise. You could spawn an entirely new instance of Sphere. So you would have two games running simultaneously, but if you minimize one and play the other you should be fine. I'm intending to do this with SSFML.


You mean an OS-level instance of Sphere? Like with `exec`?

That's an option. I for one would far prefer the engine actually be capable of starting and stopping, but so far that has been difficult.

I know that in Posix systems it would even be simple to startup a new Sphere process and assign it to a nohup parent process...hmm...
Title: Re: minisphere
Post by: Radnen on February 17, 2015, 05:45:51 pm

You mean an OS-level instance of Sphere? Like with `exec`?


Precisely.
Title: Re: minisphere
Post by: Fat Cerberus on February 18, 2015, 10:20:25 am
So out of morbid curiosity, I ran SSFML's startup game in minisphere and tried the surface fill test, one of the few things in there that actually works.  My results: ~1400 FPS.  Screenshot attached as proof. ;D

Of note: The main menu runs at around 700FPS on the same machine.

Laptop is i5-2450M with 4GB RAM, Win8.1 Pro x64.

Edit: In the interest of completeness, I also ran the test on Sphere 1.5.  It barely broke 550 FPS on the same machine.  Honestly, my experience so far has been that minisphere actually outperforms Sphere 1.5 in all cases.  At least with the standard32 plugin, anyway.  I should probably test it with the OpenGL plugin at some point...
Title: Re: minisphere
Post by: Radnen on February 18, 2015, 12:45:08 pm

So out of morbid curiosity, I ran SSFML's startup game in minisphere and tried the surface fill test, one of the few things in there that actually works.  My results: ~1400 FPS.  Screenshot attached as proof. ;D


Yep, you have better surfaces, even better than 1.5. But with HW surfaces in SSFML I could do 6500 fps on that same screen (~900 in Sphere 1.5 for comparison). I do 700 fps on that screen in SW (so worse than Sphere). But, only when many HW surfaces and features were used the fps started going way way down, it's like SFML just can't handle more than 2 or 3 surfaces on screen at the same time. I find this odd since I use hardware surfaces for tile animations in the map engine and it's really fast. But then again it's just a single surface.
Title: Re: minisphere
Post by: Flying Jester on February 18, 2015, 06:11:16 pm
The startup game is one of the worst games I know of. It does things that are exceedingly bad.

I'd be curious how FJ-GL does on your machine. I got about 600 FPS on the main menu using an ancient Dell D630.
Title: Re: minisphere
Post by: Fat Cerberus on February 18, 2015, 07:01:46 pm
I was referring to the SSFML startup game (essentially a test suite), not the default one that comes with 1.5.
Title: Re: minisphere
Post by: Fat Cerberus on February 18, 2015, 10:06:28 pm
New feature: You can change minisphere's taskbar icon by putting an image in the same directory as game.sgm with the filename game-icon.png.
Title: Re: minisphere
Post by: Fat Cerberus on February 19, 2015, 02:47:23 am
Okay so minisphere SUCKS at rendering windowstyles.  The "big window" test in Radnen's test suite?  30 fps.  Surface fill hits 1400fps easy, but it can't even maintain 60fps drawing a window?  What the hell? ???

From what I can tell, the tiling is what's doing it.  Disabling the window background as a test, it hit 1600 fps.  Which is weird, since it's just drawing the same bitmap multiple times, and I'm even batching them (unbatched, for the curious, it dropped to 3(!) fps).

This is a stumper.
Title: Re: minisphere
Post by: Radnen on February 19, 2015, 12:44:27 pm
The big window test is notoriously difficult on the graphics because it tiles a 1x1 pixel background image (in practice you won't see this). Compare to Standard32 Sphere 1.5 and SphereGL. I think SphereGL bottoms out at 2fps, or some other horrendously low number (even on my Nvidia GTX 970). In practice, try a windowstyle with a 16*16 or greater background image. Now, that said, if you are batching the image I'm really curious why you only get 30fps. Allegro must be doing some pretty naive graphics processing, despite the fact you can tell it to "defer" the drawing. I say this because you can totally do better with surfaces.
Title: Re: minisphere
Post by: Fat Cerberus on February 19, 2015, 01:40:38 pm
Standard32 Sphere manages 415 fps in the big window test (AMD A10-6800K), 13 with sphere_gl.  Note that on the same machine, this (13fps) is identical to minisphere's result.  I wonder if Allegro is using OpenGL under the hood?

No idea how SSFML pulls 3000fps in this test.  You must have one hell of a sprite batcher.
Title: Re: minisphere
Post by: Flying Jester on February 19, 2015, 02:11:19 pm
There are, I suspect, two reasons that the SphereGL driver sucks at this. The first is that sending so many vertices takes a lot of time, and bandwidth is the main limiting factor for doing hardware graphics. This is just a general issue with older OpenGL, and why using a newer version like 3.3 or 4.1 is better, because it makes doing that impossible.
The second issue is that SphereGL decides to unbind textures and disable texturing at the end of every call. This was the main place I saw for improvement when I wrote FJ-GL. There I do this lazily, just binding what ever texture I need at the start of every call, and using a 1px white texture for primitives.

I would suspect that with SphereGL, almost all of the time is spent binding, unbinding, enabling, and disabling texturing. Even if that wasn't happening, it's still deluging the GPU with vertices, which is generally not what you want to do.

An alternative would be to actually use UV coordinates to do hardware tiling of the texture, but the Sphere graphics driver API doesn't have that feature. Before I moved windowstyles to script in TS, this is what I did.
Title: Re: minisphere
Post by: Fat Cerberus on February 19, 2015, 03:06:03 pm
Hm, allegro lets me draw textured triangles via al_draw_prim().  I use this for my implementation of GradientRectangle.  I should test that later...

Oh, also abysmally slow in minisphere: GrabImage.  The SSFML image test runs at around 15fps.  Sphere 1.5 and SSFML both handle this test with aplomb.  Allegro must not like me using the backbuffer as a blit source...

Edit: I tried out the U/V hardware tiling, it seems to work pretty well, except for the SSFML Big Window test.  This is what I got.  Does the hardware not like tiling 1x1 textures?
Title: Re: minisphere
Post by: Radnen on February 19, 2015, 04:58:02 pm

Edit: I tried out the U/V hardware tiling, it seems to work pretty well, except for the SSFML Big Window test.  This is what I got.  Does the hardware not like tiling 1x1 textures?


Yes. I was not sprite-batching that 1x1 pixel, I was U/V wrapping it. Essentially I'm letting the gpu do the tiling rather than physically inputting the coords of the tiles, so I'm really only sending 4 vertices to the gpu while the rasterizer does an excellent job at tiling it. I apply this technique for the sides as well, don't forget those! That Big Window test should run really fast if done right. I'd love to use the U/V method to speed up my tile maps but it assumes it's the same image being repeated, and for tile maps the tiles change.

As for your error, are you sure you are tiling it correctly? It seems to tile at 16x16 rather than 1x1, don't use the corner size for the background image size.
Title: Re: minisphere
Post by: Fat Cerberus on February 19, 2015, 05:00:05 pm
Yeah, I got it to run at 1k+ fps, but it's not being tiled correctly.  The normal window test it tiles file, but the 1x1 background... See above.
Title: Re: minisphere
Post by: Radnen on February 19, 2015, 05:01:45 pm
ooops, you got me mid edit. I did a double check, and maybe you are tiling with the wrong parameters?

As for your lower than mine FPS, it might just be duktape?
Title: Re: minisphere
Post by: Fat Cerberus on February 19, 2015, 05:12:20 pm
Code: (c) [Select]
vertex_color = al_map_rgba(255, 255, 255, 255);
ALLEGRO_VERTEX v[] = {
{ x, y, 0, 0, 0, vertex_color },
{ x + width, y, 0, width, 0, vertex_color },
{ x, y + height, 0, 0, height, vertex_color },
{ x + width, y + height, 0, width, height, vertex_color }
};
al_draw_prim(v, NULL, bitmap, 0, 4, ALLEGRO_PRIM_TRIANGLE_STRIP);


Coord order is XYZ, UV.
Title: Re: minisphere
Post by: Radnen on February 19, 2015, 05:36:48 pm
And width and height are 1 here? I see single variables here, something has to tell it tile a 1x1 sized object in a WxH sized area.
Title: Re: minisphere
Post by: Flying Jester on February 19, 2015, 06:29:41 pm

And width and height are 1 here? I see single variables here, something has to tell it tile a 1x1 sized object in a WxH sized area.


How does Allegro do it? In OpenGL, you would just use a UV that is the number of times you want it tiled horizontally and vertically, respectively. You never need to specify the source texture's size in any way, because it's 'size' is normalized to 1.0fx1.0f before the texturing stage.
Title: Re: minisphere
Post by: Fat Cerberus on February 19, 2015, 06:41:37 pm
The code samples I've seen you specify pixels in the range of (0,0)-(w, h) for UV.  Maybe just because Allegro is designed for 2D or perhaps a remnant of its all-SW-rendered days.  I'll experiment more later.
Title: Re: minisphere
Post by: Fat Cerberus on February 19, 2015, 07:19:33 pm
https://www.allegro.cc/manual/5/ALLEGRO_VERTEX

Quote
u, v - Texture coordinates measured in pixels


So yeah, Allegro uses direct pixel offsets into the image for UV.
Title: Re: minisphere
Post by: Flying Jester on February 19, 2015, 07:48:59 pm
Ah, so it's 100% in raster coordinates, no normalization at all.

In that case, I'd expect your code to work. Maybe an Allegro issue with 1x1 textures?
Title: Re: minisphere
Post by: Radnen on February 19, 2015, 10:48:33 pm
Raw JS benchmark on my machine, in milliseconds, lower is better, in fps, higher is better (rounded to nearest ten just to look pretty):

Code: (javascript) [Select]

function game() {
var t = GetTime();

for (var i = 0; i < 10000000; ++i) {  }

Abort(GetTime() - t);
}


Sphere 1.5: 1090
MiniSphere: 472
Sphere SFML: 23

Code: (javascript) [Select]

function game() {
var fps  = 0;
var tfps = 0;
var t2   = GetTime();

while (t2 + 5000 > GetTime()) {
fps++;
Rectangle(0, 0, GetScreenWidth(), GetScreenHeight(), red);
FlipScreen();
}

Abort(fps / 5);
}


Sphere 1.5: 3000
Minisphere: 11500
Sphere SFML: 7900

Code: (javascript) [Select]

function game() {
var fps  = 0;
var tfps = 0;
var t2   = GetTime();

while (t2 + 5000 > GetTime()) {
fps++;
f.drawText(0, 0, "Hello World! Hello World! Hello World! Hello World! Hello World!");
FlipScreen();
}

Abort(fps / 5);
}


I will say minisphere draws shorter strings much faster, but it slows down longer the string. SSFML never slows down on almost any string length.

Sphere 1.5: 4900
Minisphere: 6250
Sphere SFML: 7250

Edit:
I'd say I'm rather pleased with the speed of minisphere. It draws things much faster than I had ever anticipated. I've NEVER seen an fps higher than 8500 on this machine in SSFML, but on my Win7 I can get to 10,000 with SSFML (weird, huh). So, it's awesome to see minisphere get 11,500+.

Edit 2:
In terms of space, to load the demos above:
Sphere 1.5: 3.8MB
minisphere: 8.9MB
Sphere SFML: 39.8MB (!!)

Crazy how C/C++ programs utilize so little space!
Title: Re: minisphere
Post by: Fat Cerberus on February 20, 2015, 12:14:26 am

In terms of space, to load the demos above:
Sphere 1.5: 3.8MB
minisphere: 8.9MB
Sphere SFML: 39.8MB (!!)

Crazy how C/C++ programs utilize so little space!


I assume this is RAM usage?  But yeah, I'm happy see that, outside of memory footprint (which is still tiny, let's be honest here :) ), minisphere soundly kicks Sphere 1.5's ass.  That was one of my main goals with the project, since Sphere 1.5 was really starting to show its shortcomings performance-wise with Specs, especially on less powerful machines.  Somehow, some way, Duktape, even being all-bytecode with no JIT to speak of, is STILL more than twice as fast as 1.5's Spidermonkey.

It's also good to see that you got it to build.  I was going to include the Allegro binaries in the repo, but decided it wasn't necessary since I'm just using pre-built binaries that are, at present, readily available on the web.  It's just a matter of unzipping them to the proper location.  I tried to make it as painless as possible other than that, as long as you're using Visual Studio anyway!

At some point I'm going to have to try to build minisphere on Linux.
Title: Re: minisphere
Post by: Flying Jester on February 20, 2015, 12:43:48 am
So I got minisphere building on OS X again.

Same as last time, it just locks up hard when it starts up. I know that I have no startup folder or games folder, but still...

EDIT:

Nevermind, it deadlocks even with directories nicely in place. I'll investigate.
Title: Re: minisphere
Post by: Fat Cerberus on February 20, 2015, 12:55:59 am
There's a check in place, if there's no startup folder you're supposed to get an error message like the one pictured in this screenshot.  Maybe OS X and Allegro don't get along?

Title: Re: minisphere
Post by: Radnen on February 20, 2015, 12:56:19 am
minisphere only worked with the -game parameter. I just used my Sphere Studio. Building it was easy, I just didn't know how to start running a game. The startup game had consts and so it failed, and dragging a .sgm onto the executable didn't run the game.

@LordEnglish: yes, I meant RAM. .NET apps use the CLR and .NET runtime which are memory hogs, but if I account for that I still get about 20+MB's RAM usage out of it... so it's still a memory hog (but I'm not really going to worry about that, it's just a curiosity).
Title: Re: minisphere
Post by: Fat Cerberus on February 20, 2015, 01:01:25 am

minisphere only worked with the -game parameter. I just used my Sphere Studio. Building it was easy, I just didn't know how to start running a game. The startup game had consts and so it failed, and dragging a .sgm onto the executable didn't run the game..


Does drag-and-drop work with Sphere 1.5?  Last time I tried I thought it didn't work, so I've always just opened stuff in Sphere Studio.  I'll have to implement that now.
Title: Re: minisphere
Post by: Flying Jester on February 20, 2015, 01:08:26 am
It's not the startup folder apparently, it's the system folder. I opened an issue, you are reading the system font without checking that it exists (maybe more things later, that's where it errors out).
Title: Re: minisphere
Post by: Fat Cerberus on February 20, 2015, 01:10:38 am

It's not the startup folder apparently, it's the system folder. I opened an issue, you are reading the system font without checking that it exists (maybe more things later, that's where it errors out).


Yeah, there's a gross lack of sanity checks in here.  You wonder how I implemented this all so fast, well, here's your answer. :P

I'll look into it.

Edit: Found the issue.  There are no checks whatsoever in al_load_rfn_font() despite all my other loading functions having ample error checking.  The crash is because it's trying to read from an ALLEGRO_FILE* that is NULL (because the file didn't exist).
Title: Re: minisphere
Post by: Radnen on February 20, 2015, 02:38:49 am

Does drag-and-drop work with Sphere 1.5?  Last time I tried I thought it didn't work, so I've always just opened stuff in Sphere Studio.  I'll have to implement that now.


Well, Sphere was able to do this magical thing called "registering a filetype", I couldn't seem to do this easily in .net but if I implemented drag and drop, I was able to set the Engine.exe of SSFML to open on default for any .sgm related files and it worked. I can play games in SSFML by double-clicking the .sgm game icon. This is a feature I want to add to Sphere Studio too.
Title: Re: minisphere
Post by: Flying Jester on February 20, 2015, 02:51:12 am

Well, Sphere was able to do this magical thing called "registering a filetype", I couldn't seem to do this easily in .net but if I implemented drag and drop, I was able to set the Engine.exe of SSFML to open on default for any .sgm related files and it worked. I can play games in SSFML by double-clicking the .sgm game icon. This is a feature I want to add to Sphere Studio too.


I think that was something that the installer itself did.

Drag-and-drop is much simpler, fortunately.
Title: Re: minisphere
Post by: Radnen on February 20, 2015, 02:54:10 am

I think that was something that the installer itself did.


File->Options->Register filetypes. But, it doesn't seem to list .sgm, but it does have .spk.
Title: Re: minisphere
Post by: Fat Cerberus on February 20, 2015, 02:56:03 am
That reminds me, at some point I have to add SPK support to minisphere.  That should be fun... :-\
Title: Re: minisphere
Post by: Flying Jester on February 20, 2015, 03:02:33 am
SPK support isn't that bad. It's extremely simple and uses zlib.

I would recommend for simplicity having the engine, when told to `run' an SPK, unpack the SPK into a subdirectory of your engine's root and run the game from there.

Or just don't do SPKs. It's a pretty silly format to begin with.
Title: Re: minisphere
Post by: Fat Cerberus on February 20, 2015, 10:21:58 pm

I would recommend for simplicity having the engine, when told to `run' an SPK, unpack the SPK into a subdirectory of your engine's root and run the game from there.

Or just don't do SPKs. It's a pretty silly format to begin with.


I kind of like the idea having a single file to distribute a game, and I found out I don't even have to unpack it to disk--Allegro lets me open a buffer in memory as a file (sort of like a memory mapped file but in reverse...) that I can pass to loading functions.

Of course, if we're being honest, if I weren't trying for near-1:1 Sphere compatibility I would probably just use zip instead of SPK... Well, I'll see what happens.  Either way it's not a priority for a while at least.
Title: Re: minisphere
Post by: Fat Cerberus on February 21, 2015, 02:14:23 am
(Sort of) new feature: minisphere displays internal FPS when the framerate is throttled, as by calling SetFrameRate().  This helps to differentiate skipped frames from actual slowdown.
Title: Re: minisphere
Post by: DaVince on February 21, 2015, 07:17:47 am
I still wouldn't mind having something like an spk2 format which is really just a zip or tarball you can associate with Sphere.

Nice work, by the way. And yes, you could drag an SGM over engine.exe in 1.5. Used to do it a lot. :)
Title: Re: minisphere
Post by: Fat Cerberus on February 21, 2015, 09:20:48 pm
@Radnen:
Let me just say that the SSFML testbed has been invaluable in filling out the Sphere API.  I'm almost at the point of being able to run all the non-map-engine tests, there's now only 2 or 3 that error out due to missing APIs.  Most of my implementations have gone off without a hitch and produce identical output to SSFML and Sphere 1.5 (outside of the speed tests, which seem universally faster than Sphere 1.5)--except for the surface test.

Here's the thing: I'm not actually sure what's wrong in my implementation because I can't decipher the surface test code.  It creates 4 surfaces and then proceeds to juggle them like crazy, so I don't know where minisphere is failing, but clearly something is screwed up, as you can see in the attached screenshot.  The green-tinged versions of the Blockman sprite are missing, and the whole bottom row is just the green blocks without the rotated silhouette...
Title: Re: minisphere
Post by: Flying Jester on February 21, 2015, 09:38:18 pm

Most of my implementations have gone off without a hitch and produce identical output to SSFML and Sphere 1.5 (outside of the speed tests, which seem universally faster than Sphere 1.5)--except for the surface test.


I'm curious if you've tried the version of Sphere 1.6+ that I compiled with MSVC 2010 a year ago, available at rpgmaker.net/users/FlyingJester/locker/sphere16fj.zip (http://rpgmaker.net/users/FlyingJester/locker/sphere16fj.zip). I found that simply using a newer compiler made the engine between 10% and 50% faster, depending on the situation. Using slightly newer OpenGL with FJ-GL also mad many tests--particularly texturing--up to 25% faster.

I would not be too surprised if you were still faster than that, but it would still be interesting to see the results.
Title: Re: minisphere
Post by: Radnen on February 21, 2015, 10:26:56 pm
I was doing another round of speed tests, this time using my link library.

I have a test where I compare Link.js with Lazy.js and my old Query.js functional programming libraries. I was shocked to see Query.js outpace the other by a magnitude of 10. Lazy execution is just SLOW on both minisphere and Sphere-SFML.

Code: (javascript) [Select]

// times in SSFML:
Link: 966ms
Lazy: 965ms
Query: 76ms

// times in minisphere:
Link: 2300ms
Lazy: 2298ms
Query: 132ms

// times in Sphere 1.5:
(out of memory)

// in 1.5 using 10 times less results:
Link: 300ms
Lazy: 300ms
Query: 19ms


As a reminder, Query is the library I originally made that Link is based off of. The only problem is it doesn't lazy execute the chain. Safe to say, it really is fast and kicks the crap out of the other two if you need it for a tight execution loop. In a web browser, this is not the case and is actually slower by many factors (query forces through the chain n times, where n is each added operation, Link and Lazy go through the chain once).
Title: Re: minisphere
Post by: Fat Cerberus on February 21, 2015, 10:46:33 pm

I'm curious if you've tried the version of Sphere 1.6+ that I compiled with MSVC 2010 a year ago, available at rpgmaker.net/users/FlyingJester/locker/sphere16fj.zip (http://rpgmaker.net/users/FlyingJester/locker/sphere16fj.zip). I found that simply using a newer compiler made the engine between 10% and 50% faster, depending on the situation. Using slightly newer OpenGL with FJ-GL also mad many tests--particularly texturing--up to 25% faster.

I would not be too surprised if you were still faster than that, but it would still be interesting to see the results.


minisphere still beats it, but admittedly by a much narrower margin.  Anything purely script-based though, like the Color creation benchmark, blows minisphere out of the water.  That doesn't surprise me though, since I assume this is TraceMonkey or newer and not the ancient SM in 1.5.

All 1.6FJ tests are with FJ-GL.  Testbed runs at 320x240.  CPU is an AMD A10-6800K and I'm using the GPU built into the chip (Radeon HD 8670D).

Surface Fill Rate: Fills a full-screen surface with solid green via .rectangle() and blits it.
1.6FJ: Hovers around 650 fps.
minisphere: ~2300.

Big Window: Draws a window covering nearly the full screen using a windowstyle with a 1x1 tiled background.
1.6FJ: An abysmal 15 fps.
minisphere: ~2800. (although I haven't fixed that tiling bug yet...)

Many Primitives: Fills the screen with 32x24 GradientRectangles (so 100 of them).  The one test 1.6FJ beats me at.
1.6FJ: ~3000 fps.
minisphere: ~1450.

Surfaces: Creates 4 surfaces and juggles them in odd ways.  As mentioned above, I can't really decipher the test.
1.6FJ: ~320 fps.
minisphere: ~425. (caveat emptor: minisphere doesn't render this test correctly).

Windows: Draws 3 windows using a windowstyle, two of them with a color mask, the second of which continuously expands and contracts. Note that minisphere currently ignores the colormask.
1.6FJ: ~2230 fps.
minisphere: ~2900.

So yes, minisphere is still faster at rendering, in most cases by a sizable margin. ;D
Title: Re: minisphere
Post by: Flying Jester on February 22, 2015, 07:45:57 pm

minisphere still beats it, but admittedly by a much narrower margin.  Anything purely script-based though, like the Color creation benchmark, blows minisphere out of the water.  That doesn't surprise me though, since I assume this is TraceMonkey or newer and not the ancient SM in 1.5.


It's the very last release without any JIT or tracing engine.

For a few things it's still the fastest JS engine I've seen--for example, the empty loop test with lower values (~1000000), probably since it's a very mature interpreter and isn't spending time running the code through any tracers or type analyzers, as modern SpiderMonkey does. In most real-world scenarios, this isn't the case, though.
Title: Re: minisphere
Post by: Fat Cerberus on February 23, 2015, 03:12:56 pm
Just started work on the map engine, and I have a question regarding the RMP format.  The layer header contains the following fields:

Code: [Select]
float32 parallax_x;
float32 parallax_y;
float32 scrolling_x;
float32 scrolling_y;


What I'm wondering is, what do these coordinates represent, exactly?  I know what parallax is, but I'm completely stumped as to what purpose the X/Y coordinates serve.

Also, whose idea was it to make map layers have individual width and height? ???  I can't think of a single circumstance where that makes sense.
Title: Re: minisphere
Post by: Flying Jester on February 23, 2015, 04:41:11 pm

Also, whose idea was it to make map layers have individual width and height? ???  I can't think of a single circumstance where that makes sense.

I know what parallax is

You've answered your own question :)
Title: Re: minisphere
Post by: Fat Cerberus on February 23, 2015, 04:53:06 pm
So what, it just tiles the parallax layer if it's smaller than the main layer(s)?  Besides I thought Sphere's implementation of parallax was just an image, not a real map layer.

That still doesn't answer what the parallax and scrolling X/Y in the layer header are.
Title: Re: minisphere
Post by: Flying Jester on February 23, 2015, 05:08:06 pm
No, a parallax layer could, if a different size, benefit from being larger than the main map layers.

I haven't worked with this in a while, but I'm fairly certain that parallax values control the origin when using parallax, and scrolling control the parallax speeds.

You could just poke those values with a hex editor and see the result, too. That's how I originally figured it out.
Title: Re: minisphere
Post by: Radnen on February 23, 2015, 08:15:09 pm
Parallax X/Y is a value usually default at 1. It is the mutiplier added to the camera for the parallax effect. 1, 1 is neutral, or no movement while 2 is twice as fast, etc.

Scroll X/Y is a value usually default at 0. It is the constant speed that constantly moves the background by that amount. 2 makes it scroll fast (2 pixels), while 1 is slower (the maps FPS dictates apparent speed).
Title: Re: minisphere
Post by: Fat Cerberus on February 24, 2015, 01:27:24 am
So if the scroll values are nonzero, then it scrolls continuously even if the player isn't moving?  That makes sense, then.  I guess you'd use that for a moving cloud background or such.
Title: Re: minisphere
Post by: Fat Cerberus on February 25, 2015, 02:21:12 am
I got the map engine working!  So far it can load the bitmaps from an RTS tileset and load and render a map using the proper tiles.  ChangeMap works.

Oh, and just because I'm meticulous like this: I ran minisphere and Sphere 1.5 at the same time, lined up the windows, Alt+Tabbed between them and the output is identical!  I wasn't sure if I got the alignment on the sprites right or not, but it looks like I did.
Title: Re: minisphere
Post by: Radnen on February 25, 2015, 02:35:31 am
Awesome! The map is the most daunting part. It's not hard to do but there's certainly a lot of pieces in it. To emulate Sphere correctly you'll basically need to copy it's line intersection code for obstructions and the command queue code for movement. I'd be impressed if you manage to get lithonite.js working the way it was intended. (In SSFML it works mostly correctly besides a few edge cases... with edges!)

If you are drawing the layers to surfaces and then drawing the surface to screen, check to make sure you can do tile animations easily and fast enough. That'll be the next big step.
Title: Re: minisphere
Post by: Fat Cerberus on February 25, 2015, 09:55:21 am
I didn't even bother with the extra complexity of surfaces.  I just told Allegro to defer drawing and drew all the visible tiles directly to the backbuffer.  See, I looked into how the deferred drawing works, and what benefits most from it is bitmap drawing--when you defer, Allegro batches all al_draw_bitmap() calls and then sends them all to the GPU at once as a single, huge triangle list.  In theory this should be blazing fast without the need to cache the layers on a surface.  Even more so once I implement atlasing for tilesets (currently each tile gets its own bitmap).
Title: Re: minisphere
Post by: Radnen on February 25, 2015, 12:19:03 pm

I didn't even bother with the extra complexity of surfaces.  I just told Allegro to defer drawing and drew all the visible tiles directly to the backbuffer.  See, I looked into how the deferred drawing works, and what benefits most from it is bitmap drawing--when you defer, Allegro batches all al_draw_bitmap() calls and then sends them all to the GPU at once as a single, huge triangle list.  In theory this should be blazing fast without the need to cache the layers on a surface.  Even more so once I implement atlasing for tilesets (currently each tile gets its own bitmap).


Ok, awesome! That actually will make animations easier to do. Tileset atlasing would also end up as a huge win, too. Allegro would be using OpenGL, and sending different texture pointers to HW is certainly a bottleneck.
Title: Re: minisphere
Post by: Fat Cerberus on February 25, 2015, 03:19:43 pm
Okay, whose idea was it to use line segments for obstruction? >:(. I know it's more flexible than bounding-box collision, but the crazy math required to check for intersections is insane.  Nothing like rects where it's just a few comparisons and you're done.  I looked at Sphere's source code and the TestSegments function alone must be at least 100 lines of code...
Title: Re: minisphere
Post by: Radnen on February 25, 2015, 04:29:45 pm
Line intersect is faster than pixel-perfect and can be nearly as accurate (if you map a decent contour), while bounding-box can be clunky on organic objects. That said my implementation of line intersect code is definitely not 100 lines, but then again I did abstract things away into helper classes. But even then, a hundred lines for that is a lot.

https://github.com/Radnen/sphere-sfml/blob/master/Engine/Engine/Objects/Line.cs#L33-49

The math there is the core of it, I'm sure you can abstract lines into a struct in C and make a few helper functions. It's literally that, and for loops, checking all the lines, whether the source is a person, a tile, or a custom map obstruction line.
Title: Re: minisphere
Post by: Flying Jester on February 25, 2015, 06:48:39 pm
It's not that crazy (https://github.com/FlyingJester/TurboSphere/blob/master/bin/system/scripts/turbo/segment.js#L49), or at least it shouldn't have be. My JS implementation is totally contained inside that one file, and you'll note it's only 109 lines :)

Eventually, Chad Austin had wanted to use BSP trees for obstructions. I suspect that was a part of the change to segments.
Title: Re: minisphere
Post by: Fat Cerberus on February 25, 2015, 09:16:35 pm
Yeah, wow, yours and Radnen's are so much shorter and more elegant.  I just checked the code in Sphere again, and this is what it looks like:
https://github.com/sphere-group/sphere/blob/master/sphere/source/common/ObstructionMap.cpp#L63-L189

Even accounting for blank lines and comments, that function still has to be close to 100 lines of code.
Title: Re: minisphere
Post by: Fat Cerberus on March 01, 2015, 11:38:30 am
Look what works in minisphere! ;D  Well, at least up till ExecuteGame is called...
Title: Re: minisphere
Post by: Fat Cerberus on March 04, 2015, 10:59:19 am
In the Steps of the Blackfoot is almost completely functional (barring a couple glitches and lack of person-tile collision) in minisphere! :D
Title: Re: minisphere
Post by: DaVince on March 05, 2015, 10:57:18 am
Wow, nice work man. :) Is performance decent?
Title: Re: minisphere
Post by: Fat Cerberus on March 05, 2015, 11:54:08 am

Wow, nice work man. :) Is performance decent?


It performs better than Sphere 1.5 in nearly all cases. ;D
Title: Re: minisphere
Post by: Fat Cerberus on March 07, 2015, 02:11:27 am
This just in: minisphere fully supports obstruction!

At this point I think minisphere is just about ready for a public beta release.  There are still a bunch of unimplemented APIs, but even as-is many existing Sphere games run with minimal--sometimes no--modification, so no real sense holding back anymore.  So get ready, minisphere is coming! ;D

Or, you know, you could just check it out from GitHub now and build it yourself, but... ;)

update: Aquatis now runs up to the intro cinematic before erroring out due to lack of Surface:applyLookup().  So close...
Title: Re: minisphere
Post by: Fat Cerberus on March 09, 2015, 02:25:56 am
Just released the first public beta of minisphere! ;D  Try it out!  Link is in the OP, but I'll repost it here for convenience.

Current release: minisphere 1.0-b1 (March 9, 2015)
Download on Google Drive (https://drive.google.com/file/d/0BxPKLRqQOUSNcjFBdXZ2dDNhd2M/view?usp=sharing) - GitHub (https://github.com/fatcerberus/minisphere)
Title: Re: minisphere v1.0-b1
Post by: Fat Cerberus on March 10, 2015, 01:37:51 am
So, a few new features:

GetExtensions() returns a list of strings representing the engine's capabilities.  For example, calling this in minisphere currently returns:
[ "sphere-legacy", "minisphere" ]

I got the idea after skimming the Pegasus stuff on GitHub.

The other new feature is an optional stack offset passed to Abort(), like so:
Code: (javascript) [Select]
Abort("ThisFunctionSucks(): This function sucks, why did you call it?", -1);


It allows you to blame the error on a script further up the call stack, useful in libraries to report invalid parameters or other such errors.  If the stack offset is not provided, it defaults to 0, meaning "location of Abort call" - i.e. the normal behavior.
Title: Re: minisphere v1.0-b1
Post by: Radnen on March 10, 2015, 02:04:39 am

It allows you to blame the error on a script further up the call stack, useful in libraries to report invalid parameters or other such errors.  If the stack offset is not provided, it defaults to 0, meaning "location of Abort call" - i.e. the normal behavior.


That's really awesome! In RadLib I had to do some SM only hacks to get line numbers and report the penultimate function call in the call stack. This would undoubtedly make that process much easier under your engine. +1
Title: Re: minisphere v1.0-b1
Post by: Fat Cerberus on March 10, 2015, 07:13:36 pm
See, the whole thing was kind of a happy accident - by default Duktape reported Abort-generated exceptions as coming from api.c, since that's where the duk_error() call is.  In order to get it to show the script file that aborted instead, I had to call into Duktape to walk a step or two up the stack and then use duk_error_raw, which lets you override the file name and line number of the thrown exception.  Of course, since I now had full access to the callstack, well... :)
Title: Re: minisphere v1.0-b1
Post by: Flying Jester on March 10, 2015, 07:18:51 pm
Just before I switched to SM, TurboSphere reported the entire JS stack on crash. Unfortunately, V8 couldn't give line numbers for higher frames, but it was still nice to see.

Would that be possible/not-horribly-complicated with Duktape?
Title: Re: minisphere v1.0-b1
Post by: Fat Cerberus on March 10, 2015, 07:25:50 pm
Trivial.  The stack offset on Abort isn't required to be -1, it can go all the way up the stack if needed and even gives line numbers.  The functionality I use to do it--Duktape.act()--is even exposed to script, so as long as minisphere uses Duktape, the entire callstack will available to scripts at any time.

I have to say, Duktape doesn't look like much, but it's actually a pretty awesome little JS engine.  It definitely took me by surprise.
Title: Re: minisphere 1.0b1
Post by: Fat Cerberus on March 10, 2015, 09:01:22 pm
Just implemented atlasing for RFN fonts.  The original implementation used individual ALLEGRO_BITMAPs for each glyph, which I suspect was the main reason for minisphere's horrible text-rendering performance up to now.  The drag became especially noticeable in Spectacles when the console was displayed.  Now it seems to be much better!
Title: Re: minisphere 1.0b1
Post by: Radnen on March 10, 2015, 09:23:34 pm

Just implemented atlasing for RFN fonts.  The original implementation used individual ALLEGRO_BITMAPs for each glyph, which I suspect was the main reason for minisphere's horrible text-rendering performance up to now.  The drag became especially noticeable in Spectacles when the console was displayed.  Now it seems to be much better!


Awesome!

Were you able to figure out the windowstyle issue? I know it may break on 1x1 sized images, but what about 2x2 or larger?
Title: Re: minisphere 1.0b1
Post by: Fat Cerberus on March 10, 2015, 09:28:17 pm
See, the odd thing is, that big window test?  It displays fine if I force Allegro into OpenGL mode.  It's only with D3D (Allegro's default under Windows) that the issue shows up.  I'll have to test with larger backgrounds though.  Honestly I haven't really worried about it since nobody's going to be using 1x1 windowstyles in a real game and the more typical sizes seem to render fine.
Title: Re: minisphere 1.0b1
Post by: Fat Cerberus on March 11, 2015, 03:14:20 am
So I just finished implementing zones, which completes the map engine. ;D. Still missing APIs I'm sure, but the map engine itself is, as far as I can tell, a complete functional recreation of Sphere's.

Now... what exactly do I need to do to get Lithonite working?  It seems to work somewhat in minisphere, but many of the zone types are completely non-functional and others are glitchy--for example slopes cause me to get stuck in diagonal movement eternally, and jumping off ledges makes my guy fly off the map completely out of my control.  Since the APIs Lithonite needs are all implemented and should be behaving identically to Sphere, I fail to understand what is causing the glitches.

Also: Lithonite reminded me that I never actually implemented tile animation... I should get on that!
Title: Re: minisphere 1.0b1
Post by: Fat Cerberus on March 11, 2015, 09:41:16 am
So that didn't take long--I got Lithonite working! :D  Little-known fact about SetDelayScript(): Despite the name of the function, it actually queues scripts, rather than setting one at a time.  Not supporting this was the reason it didn't work for me, and likely the reason it doesn't work properly in SSFML either--it only tracks one delay script at a time.
Title: Re: minisphere 1.0b1
Post by: Radnen on March 11, 2015, 08:10:12 pm

Not supporting this was the reason it didn't work for me, and likely the reason it doesn't work properly in SSFML either--it only tracks one delay script at a time.


Good to know!

Also I really need to implement more of the API esp. zone and trigger execution nuances, etc. I just need to find the time sometime to do it.
Title: Re: minisphere 1.0b1
Post by: Fat Cerberus on March 11, 2015, 11:39:30 pm

Also I really need to implement more of the API esp. zone and trigger execution nuances, etc. I just need to find the time sometime to do it.


I have to say, recreating Sphere like this, I have a new respect for the built-in map engine.  I always wrote it off because of the all the bugs in the Sphere 1.x implementation, but now I think it's pretty great.  The fact that minisphere fixes a lot of the aforementioned bugs (some of them without conscious effort on my part even) makes it that much more awesome.
Title: Re: minisphere 1.0b1
Post by: Fat Cerberus on March 12, 2015, 12:32:41 am
What's the best way to implement tile animation?  My current implementation just does a brute-force traversal of the map, counting frames for each tile and switching them in-place, but I have a feeling this is going to end up being a performance drain on larger maps.  I looked at both SSFML and Sphere's source, but I can't make heads or tails of the lookup method either one uses.  Care to enlighten me?

Edit: Okay, I really need to shut up. :-X  Leave me to my own devices for an hour and I figure stuff out every single time.  Instead of traversing the map, I set up a system where I walk the tileset and, at render time, remap the tile indices passed in from the map engine to their current animation state.  Funny thing about it: I ended up looking at the Sphere source again after doing this, and it turns out Sphere uses this exact same system, the code for it is just a lot sloppier.
Title: Re: minisphere 1.0b1
Post by: DaVince on March 12, 2015, 03:42:07 pm
Wow, Lord English. You're making amazing progress with this. I wish I had enough time to check it out!
Title: Re: minisphere 1.0b1
Post by: Radnen on March 12, 2015, 08:05:25 pm

Edit: Okay, I really need to shut up. :-X  Leave me to my own devices for an hour and I figure stuff out every single time.  Instead of traversing the map, I set up a system where I walk the tileset and, at render time, remap the tile indices passed in from the map engine to their current animation state.  Funny thing about it: I ended up looking at the Sphere source again after doing this, and it turns out Sphere uses this exact same system, the code for it is just a lot sloppier.


Sane here. I walk the tileset, find the correct timings (just once), and then just use them at the appropriate time intervals. I only update the timings when the user sets the tile animation in code.

BTW I just want to add between you me and FJ, working on our own engines has really helped us see how Sphere works internally and I'd say we'd all know at this point how to hack sphere into submission JS code-side. ;)
Title: Re: minisphere 1.0b1
Post by: Fat Cerberus on March 12, 2015, 11:40:15 pm
Speaking of tilesets, I just implemented tile atlasing!  My refcounted image system really helped here, I just have it create subimages off of the atlas, then free the atlas--when the last subimage (tile) is freed, the atlas gets freed automatically.  The best part is that the subimages are normal image_t pointers, so the atlas doesn't even interfere with things like SetTileImage, which can just free and replace the image pointer as normal.
Title: Re: minisphere 1.0b1
Post by: Fat Cerberus on March 14, 2015, 01:28:03 am
Well, minisphere has been a far bigger success than I ever anticipated.  Not only does it beat Sphere 1.5 in performance hands-down, but it runs the vast majority of existing Sphere games better than Sphere does.  I've known Sphere was buggy for quite a while, but I never realized it was quite this bad until I started comparing the two engines side-by-side later in development.

This is the part where I'll be honest: minisphere's development was largely motivated by self-interest.  After I finished the Spectacles battle engine, I started work on the field, and then realized that Sphere's map engine doesn't react well at all to blocking logic, UpdateMapEngine be damned.  For the longest time I thought it was a bug in the Specs threader, but no, it turns out it has something to do with Sphere's frameskipping logic causing it to skip a ton of FlipScreens (exactly equal to the number of frames spent in the MapEngine loop, interestingly enough), which I didn't fully realize until I got minisphere's map engine up and running and the threader behaved just fine there.

Whatever the case, I'm confident to say that the version of minisphere as it exists on GitHub right now is quite suitable as a drop-in replacement for Sphere 1.5.  I'll be posting minisphere 1.0b2 soon.  It would be a release candidate instead of a beta, but I haven't implemented networking support yet.  I want to at least do that before going gold with 1.0.
Title: Re: minisphere 1.0b1
Post by: Flying Jester on March 14, 2015, 06:03:46 am

This is the part where I'll be honest: minisphere's development was largely motivated by self-interest.


So was TurboSphere. I was frustrated by not being able to do N-body physics and 3D matrix transformations in script, and I was disappointed in the polygon drawing performance.
The best engines solve actual problems that you run into :)
Title: Re: minisphere 1.0b1
Post by: Fat Cerberus on March 14, 2015, 10:30:23 am
New API function: IsSkippedFrame().  This returns true if the next FlipScreen will be ignored, allowing the game to skip rendering for a frame to maximize the effectiveness of frame skipping.  The engine already does this internally for primitives and blits (except for render-to-surface, which is exempt for obvious reasons), so figured I may as well expose it to script. :)
Title: Re: minisphere 1.0b1
Post by: N E O on March 15, 2015, 12:36:44 am
Since you've implemented the map engine, can you run my NShoot demo, Artyxx (https://github.com/apollolux/artyxx) in minisphere? I want to find out if the issues it had in vanilla 1.5 still exist, since I know that I also had sloppy code at the time (e.g. wrote an unoptimized particle system script NPart b/c the beta ParticleSystem didn't have collision or the ability to access individual particles for me to script collision myself) but was running against a whole bunch of map engine ridiculousness too.
Title: Re: minisphere 1.0b1
Post by: Fat Cerberus on March 15, 2015, 01:57:45 am
Hm, seems I can't even get past the intro.  minisphere fails with "invalid base value" (I haven't located the cause yet, but basically this is Duktape's obtuse way of calling out attempts to use undefined as an object--probably due to an API I haven't implemented yet), and Sphere 1.5 fails at the same point with an error on SetLayerRenderer (invalid integer "Base" for argument 1).
Title: Re: minisphere 1.0b1
Post by: Fat Cerberus on March 15, 2015, 02:40:39 am
So one thing that always bugged me about the Sphere map engine is that maps smaller than the screen are rendered aligned to the top-left corner of the screen.  Well, I've fixed that: Small maps will be centered in minisphere, as shown in the screenshot below.

Edit: Also, just because of how awesome this is: One of my most recent changes was to massively improve the CPU usage of minisphere under framerate throttling.  As you can see in the second screenshot here, my 2nd gen i5 is barely breaking a sweat running minisphere at 60fps without skipping a single frame.  See, when there's time left over after a frame, instead of uselessly spinning, the process actually goes to sleep via al_wait_for_event_timed(), greatly reducing its CPU footprint.  Of course, this doesn't apply if framerate throttling is off--at that point there's obviously nothing I can do.
Title: Re: minisphere 1.0b1
Post by: Radnen on March 15, 2015, 04:36:47 am
Heh, you know most of the awesomeness with minisphere has a lot to do with, well... allegro being awesome. :P

This truly is a light-weight and fast implementation of Sphere! Can't wait to see full parity with the Sphere catalog. And as a bonus since allegro is so awesome there might be a future when minisphere get's a more decent engine like V8, but in the meantime this simply seems good enough.
Title: Re: minisphere 1.0b1
Post by: Fat Cerberus on March 15, 2015, 10:31:25 am

And as a bonus since allegro is so awesome there might be a future when minisphere get's a more decent engine like V8, but in the meantime this simply seems good enough.


Yes, unfortunately Duktape is the weak link here.  For what it is (a lightweight, easy-to-embed JS engine, basically the JS answer to Lua) it's great, but there are some issues.  Namely, compiling scripts, even tiny delayscripts, is slow, even compared to Sphere 1.5's SpiderMonkey.  This means that anything that abuses delayscripts--like, say, Lithonite--is going to take a pretty noticeable performance hit.  Of course this can be alleviated in new games if I allow SetDelayScript and friends to take a (already-compiled) function as argument, but this does nothing to help games using the older method.

Not only that, but games with large codebases, for instance Trial & Error and Aquatis, cause a pretty painful delay on startup while all the scripts are compiled.
Title: Re: minisphere 1.0b1
Post by: Fat Cerberus on March 16, 2015, 12:52:39 am
So I was going to wait until after releasing 1.0b2 to do this, but I had some free time today and implemented the networking API.  For this I used a tiny asynchronous networking library called Dyad (https://github.com/rxi/dyad), which is essentially just a thin wrapper over the target platform's native sockets implementation (e.g. Winsock for Windows, BSD sockets for other platforms) and adds only about 3k to the engine executable's size.

I tested it out with this code:
Code: (javascript) [Select]
var socket = OpenAddress("www.google.com", 80);
while (!socket.isConnected()) FlipScreen();
socket.write(CreateByteArrayFromString("GET / HTTP/1.1\r\nHost: www.google.com\r\n\r\n"));
while (socket.getPendingReadSize() == 0) FlipScreen();
Abort(CreateStringFromByteArray(socket.read(socket.getPendingReadSize())));


And got some HTML code back in both Sphere and minisphere, so it works perfectly!
Title: Re: minisphere 1.0b2
Post by: Fat Cerberus on March 16, 2015, 02:07:59 pm
minisphere 1.0b2 is out. ;D
Title: Re: minisphere 1.0b2
Post by: DaVince on March 17, 2015, 08:36:26 am
"Oh, I just casually implemented networking inbetween minor versions." Damn, man. ;D
Title: Re: minisphere 1.0b2
Post by: Fat Cerberus on March 17, 2015, 11:34:02 am
Haha, what can I say, I'm very motivated.  :D. For what it's worth, I'm getting very close to a 1.0 release, but I need some outside testing to happen first, particularly with regard to joystick support, which I couldn't test myself as I don't have a gamepad handy.

I'm thinking I might need to make some enhancements to the networking API, though.  As implemented in Sphere (and minisphere), it's very barebones and lacks basic functionality--for example, there's no way to tell from script when a socket has finished receiving data.
Title: Re: minisphere 1.0b2
Post by: Flying Jester on March 17, 2015, 05:33:49 pm
I think the networking API is very close to what it should be. I rather think it should basically be the same as the BSD socket API, which it almost is. Mainly, I believe that there should be an "Accept" function in addition to "ListenOnPort".

I'm not sure what you mean by "finished receiving data".

Is the SCons file commit going to be pulled in?
Title: Re: minisphere 1.0b2
Post by: Fat Cerberus on March 17, 2015, 05:45:14 pm
Ignore me on the finished receiving thing.  I didn't do enough research into how sockets work to realize that there was no built-in facility for the receiver to know when the other end has sent a full response (for instance, a full HTML doc), it's just an open data channel.

...and yes, that means I somehow successfully implemented networking without knowing a thing about how sockets work.  :P

As for SCons, can I cherry-pick only the SCons stuff somehow?  The rest of the commits in that pull would probably be a very bad idea to merge in at this point.  I may just have to close the pull and import them manually.
Title: Re: minisphere 1.0b2
Post by: Flying Jester on March 17, 2015, 09:32:18 pm

Ignore me on the finished receiving thing.  I didn't do enough research into how sockets work to realize that there was no built-in facility for the receiver to know when the other end has sent a full response (for instance, a full HTML doc), it's just an open data channel.

...and yes, that means I somehow successfully implemented networking without knowing a thing about how sockets work.  :P

As for SCons, can I cherry-pick only the SCons stuff somehow?  The rest of the commits in that pull would probably be a very bad idea to merge in at this point.  I may just have to close the pull and import them manually.


I do know there is a way to cherry-pick certain commits, using a command literally called `git cherrypick', but I don't recall exactly how it works. I haven;t had to do that since last summer.

I can just make up some new SCons files. There are new source files to account for, anyway. It's probably easiest that way.

Re sockets, you are guaranteed that write commands are not broken when they are read on the other end, even if the amount sent is greater than what a literal packet can hold. You are only in danger of having multiple responses queued up and not being able to distinguish where one ends and another begins. This is almost always solved by the application.

And I believe that anything more is totally up to the application, and really should only be in script in a Sphere-like environment. Only holding the BSD API allows Sphere-like engines to talk to any kind of server and be any kind of server. Any extension of this by the engine in native, I believe, puts this in danger.
Title: Re: minisphere 1.0b2
Post by: Fat Cerberus on March 18, 2015, 12:12:14 am
So I see what you mean about the lack of any kind of Accept() function for listening sockets.  As it's implemented in Sphere, a socket created via ListenOnPort is literally replaced by the first thing to connect to it, meaning trickery like this is required to run an actual server:

Code: (javascript) [Select]
Threads.doWith({ socket: ListenOnPort(80) },
function() {
if (this.socket.isConnected()) {
var text = GetVersionString();
var data = CreateByteArrayFromString("HTTP/1.1 200 OK\r\nContent-Length: " + text.length + "\r\n\r\n" + text);
this.socket.write(data);
this.socket.close();
this.socket = ListenOnPort(80);
}
return true;
}
);


(Also, let's just take a moment to appreciate how awesome the Specs threader is. :) )
Title: Re: minisphere 1.0b2
Post by: Fat Cerberus on March 18, 2015, 10:03:49 pm
So I made some improvements to the networking API as suggested by Jester:

ListenOnPort() now takes an optional second parameter, max_backlog.  If not provided, it defaults to zero, which causes the socket to behave as in Sphere 1.5, closing the listener and replacing the socket in-place with that of the first client connection.  If max_backlog is greater than zero, it acts as a BSD socket, and the script must call the new Socket:acceptNext() method to get socket objects for new client connections.  If acceptNext() is not called periodically, any pending connections past the size of the backlog will be automatically dropped.  The listening socket remains open as a listener until closed.

The way this was done, it's fully backwards compatible with the Sphere API and only adds one new method. :)

Edit: Okay, I fibbed.  There are 3 new methods: The aforementioned acceptNext(), getRemoteAddress() and getRemotePort().  It's good to know who's connecting to your server, methinks. :P
Title: Re: minisphere 1.0b3
Post by: Fat Cerberus on March 19, 2015, 03:22:18 pm
Just posted minisphere 1.0b3.  See OP for download link and changelog.

At this point I have to recommend Sphere developers use minisphere over vanilla Sphere if possible.  Only a few APIs are unimplemented (particularly oddball stuff like color matrices), and minisphere is not only faster, but in my testing at least, less glitchy than Sphere 1.5.

Oh, and NEO: After a few necessary minor edits to the code, your Artyxx demo works great in minisphere. 8)
Title: Re: minisphere 1.0b3
Post by: Radnen on March 19, 2015, 08:12:57 pm

At this point I have to recommend Sphere developers use minisphere over vanilla Sphere if possible.  Only a few APIs are unimplemented (particularly oddball stuff like color matrices), and minisphere is not only faster, but in my testing at least, less glitchy than Sphere 1.5.


I will say the only outstanding issues are discrepancies between SpiderMonkey and ECMA5.1, such as const and error constructors and other wacky additions Mozilla decided to add. Since my RadLib library has lots of this intrinsic stuff I'll have to spend time converting it. But yes, I should update my code rather because ECMA complacent JS ideally works anywhere ECMA is supported: it's the smallest subset that's considered correct JS.
Title: Re: minisphere 1.0b3
Post by: Radnen on March 19, 2015, 08:18:21 pm
I'm going to log bugs on github, even if they can't be fixed (such as const).
Title: Re: minisphere 1.0b3
Post by: Fat Cerberus on March 19, 2015, 08:21:27 pm
I actually tried to fix the const thing early on by editing Duktape, but it's lexer code is... Obtuse.  Go ahead, try to find keywords in duktape.c, they just aren't there.  It baffles me.
Title: Re: minisphere 1.0b3
Post by: Fat Cerberus on March 20, 2015, 12:27:41 pm
Good news: I posted an issue on Duktape's GitHub repo regarding the lack of a reported filename for syntax errors, and the dev fixed it.  So now minisphere reports the exact location of syntax errors, making it easier to locate consts that need to be changed. :)
Title: Re: minisphere 1.0b2
Post by: Flying Jester on March 20, 2015, 09:08:56 pm

So I made some improvements to the networking API as suggested by Jester:

ListenOnPort() now takes an optional second parameter, max_backlog.  If not provided, it defaults to zero, which causes the socket to behave as in Sphere 1.5


It defaults to 16 in Sphere 1.5. That's just the number of pending connections that have not yet been Accept'ed before connections start getting refused outright.
Title: Re: minisphere 1.0b3
Post by: Fat Cerberus on March 20, 2015, 09:26:07 pm
That makes no sense because Sphere 1.5 has no accept() method for listeners and (from what I can tell) the listening socket closes as soon as someone connects to it, at which point the object becomes (i.e. is internally mutated into) a normal bidirectional socket.  Sphere might set a backlog of 16 internally, but the way the API is exposed, it effectively can only accept a single connection.  Hence the zero default in minisphere to specify the legacy behavior.

Note that the actual backlog on a socket is likely going to be larger than whatever you pass to ListenOnPort() for the second parameter anyway because Dyad is set up to accept() connections automatically--I get a callback with the new socket only after this has been done for me.  Which means minisphere can't actually reject any connections--it has to disconnect them after they've already been established if its own backlog is full.
Title: Re: minisphere 1.0b3
Post by: Radnen on March 20, 2015, 09:51:10 pm
Do connections close properly when Sphere shuts down? One thing I noticed is that when I made networked games in Sphere 1.5, and you hit the X button or Alt+F4 out of the app, you couldn't close open files, connections, etc.

In fact it'd be nice to have access to that as a new feature. The "on close" or "shutdown" callback.
Title: Re: minisphere 1.0b3
Post by: Fat Cerberus on March 20, 2015, 10:02:52 pm
All native-implemented objects in minisphere have finalizers and Duktape guarantees they will be called on shutdown if the GC hasn't done so already.  Duktape also uses refcounting in addition to mark-and-sweep, so in most cases you don't even have to wait for an object to be collected when it goes out of scope--it's finalized immediately.  Short of a segfault (or refcounting bug in minisphere--I've encountered these before and they're not pretty), you're not likely to leak resources in minisphere.

That said, a shutdown callback could be useful.  I'll consider implementing such a thing.
Title: Re: minisphere 1.0b3
Post by: Fat Cerberus on March 21, 2015, 01:21:50 pm
I've decided to make minisphere's API functions more JS engine agnostic.  They are now defined like this:

Code: (c) [Select]
static js_retval_t
js_CreateColor(_JS_C_FUNC_ARGS_)
{
js_begin_api_func("CreateColor");
js_require_num_args(3);
js_int_arg(1, r_val);
js_int_arg(2, g_val);
js_int_arg(3, b_val);
js_maybe_int_arg(4, a_val, 255);

r_val = fmin(fmax(r_val, 0), 255);
g_val = fmin(fmax(g_val, 0), 255);
b_val = fmin(fmax(b_val, 0), 255);
a_val = fmin(fmax(a_val, 0), 255);
js_return_sphereobj(color, al_map_rgba(r_val, g_val, b_val, a_val));
}


Unlike Sphere's beginfunc() macro, this still looks like a normal C function and so doesn't hurt the code's readability.

The thing is though, I don't know if I'm up for the massive refactoring that's going to be required to get all my API functions to conform to this setup... :-\
Title: Re: minisphere 1.0b4
Post by: Fat Cerberus on March 22, 2015, 10:26:57 am
minisphere 1.0b4 is up.  No new APIs this time around, just under-the-hood changes.  Much of the architecture was rewritten, however (I now use fopen instead of Allegro's file routines, for example), so I thought it was good to push out another release so I could get some testing in.
Title: Re: minisphere 1.0b4
Post by: N E O on March 22, 2015, 07:21:39 pm

Oh, and NEO: After a few necessary minor edits to the code, your Artyxx demo works great in minisphere. 8)


Groovy! Can you PR those edits to the repo?
Title: Re: minisphere 1.0b4
Post by: Fat Cerberus on March 22, 2015, 09:48:39 pm


Oh, and NEO: After a few necessary minor edits to the code, your Artyxx demo works great in minisphere. 8)


Groovy! Can you PR those edits to the repo?


Haha, sure, thing. :)  Until you said that I had actually forgotten I checked it out from GitHub!
Title: Re: minisphere 1.0b3
Post by: Flying Jester on March 23, 2015, 01:28:09 am

That makes no sense because Sphere 1.5 has no accept() method for listeners and (from what I can tell) the listening socket closes as soon as someone connects to it, at which point the object becomes (i.e. is internally mutated into) a normal bidirectional socket.  Sphere might set a backlog of 16 internally, but the way the API is exposed, it effectively can only accept a single connection.  Hence the zero default in minisphere to specify the legacy behavior.


That's not what that number means. Setting that number to zero in Linux or BSD will result in no connections whatsoever being accepted. If any are accepted with winsock, that is a bug in winsock or a misinterpretation of the BSD API. Setting that number sets the number of connections that can exist waiting on listening socket, waiting for an Accept. Setting it to zero means that there can be zero sockets ready to be accepted, the queue has a size of zero and will always be empty.

Setting it to one is much more correct.
Title: Re: minisphere 1.0b3
Post by: Fat Cerberus on March 23, 2015, 01:37:34 am


That makes no sense because Sphere 1.5 has no accept() method for listeners and (from what I can tell) the listening socket closes as soon as someone connects to it, at which point the object becomes (i.e. is internally mutated into) a normal bidirectional socket.  Sphere might set a backlog of 16 internally, but the way the API is exposed, it effectively can only accept a single connection.  Hence the zero default in minisphere to specify the legacy behavior.


That's not what that number means. Setting that number to zero in Linux or BSD will result in no connections whatsoever being accepted. If any are accepted with winsock, that is a bug in winsock or a misinterpretation of the BSD API. Setting that number sets the number of connections that can exist waiting on listening socket, waiting for an Accept. Setting it to zero means that there can be zero sockets ready to be accepted, the queue has a size of zero and will always be empty.

Setting it to one is much more correct.


I know what the backlog is.  I researched sockets after implementing the networking API to better understand how everything works.

Here's the thing: The Dyad library I'm using for networking manages it for me.  From what I can tell, Dyad by default sets a backlog of 511 and accepts connections for me, it's just that, for compatibility reasons, I defined zero passed to ListenOnPort() (NOT winsock) to be a sentinel meaning "make this socket behave as in Sphere 1.5".  It doesn't mean the actual backlog is zero--that would be nonsensical!

Long story short: minisphere still accepts connections past the backlog value passed to ListenOnPort, it just drops them immediately if it's own backlog is full.  Understand now?
Title: Re: minisphere 1.0b4
Post by: Fat Cerberus on March 24, 2015, 11:00:24 pm
New APIs: RoundRectangle and OutlinedRoundRectangle.  No idea why Sphere didn't have these, but now minisphere does!
Title: Re: minisphere 1.0b4
Post by: Fat Cerberus on March 25, 2015, 12:45:01 am
Hm, looks like minisphere was so successful that I attracted a spambot! :P
Title: Re: minisphere 1.0b4
Post by: Fat Cerberus on March 25, 2015, 02:57:56 am
Okay, so heads up: Next release of minisphere will include at least the following as system scripts:



I may add more, but these five seemed like good candidates for essential functionality to include in the default distribution.

I'm removing all the old legacy cruft that the system scripts folder comes with in vanilla.  I haven't seen a game yet that uses any of them, and at least half of them are broken or incompatible now anyway.  They're just dead weight at this point.
Title: Re: minisphere 1.0b4
Post by: Flying Jester on March 25, 2015, 09:39:08 pm
The three that I know are used at least sometimes are circles.js, time.js, and colors.js. I would leave at least those. Colors just needs a working CreateColor (I modified the one included with TurboSphere to use the color constructor), time.js just needs GetTime(), and circles.js needs Surfaces that have the circle primitive methods.

On the other hand, I really recommend exposing a Delay function that replaces time.js from the engine itself, as TurboSphere does, to allow games to not use 100% of the CPU even when idling.
Title: Re: minisphere 1.0b4
Post by: Fat Cerberus on March 26, 2015, 03:21:01 am

On the other hand, I really recommend exposing a Delay function that replaces time.js from the engine itself, as TurboSphere does, to allow games to not use 100% of the CPU even when idling.


Way ahead of you on that one.  I already implemented Delay() a few days ago, it will be in 1.0b5.  Keeping CPU usage down is one of my main concerns with minisphere.  I hate that Sphere 1.5 maxes out a core the whole time it's running, which is why I implemented the timed sleep after FlipScreen a few betas ago.
Title: Re: minisphere 1.0b4
Post by: Fat Cerberus on March 26, 2015, 12:54:51 pm
So, minisphere finally has in-engine error messages.  Having to grab the mouse when an error popped up during testing was starting to get annoying. :P  See screenshot below:

Edit: minisphere 1.0b5 is out!  This will probably be the last beta, next release will either be 1.0 or a release candidate, depending on how confident I'm feeling about the code.
Title: Re: minisphere 1.0b5
Post by: Fat Cerberus on March 26, 2015, 05:24:49 pm
Oh, by the way, there's an Easter egg related to the Abort and Alert functions, let's see if anyone can find it.  :) (No cheating by looking at the code!)
Title: Re: minisphere 1.0
Post by: Fat Cerberus on March 28, 2015, 03:48:39 pm
Well, after 2 months of nonstop coding, minisphere 1.0 has been released! ;D  There's still a handful of Sphere APIs missing or unimplemented, but no real showstoppers, so I figured now was as good a time as any for the 1.0 release.

See the OP to download and go crazy!  Don't forget to report bugs as I'm sure there's a ton of them... :P
Title: Re: minisphere 1.0.1
Post by: Fat Cerberus on March 29, 2015, 01:45:02 am
Just posted minisphere 1.0.1, fixing a couple bugs discovered by Radnen.
Title: Re: minisphere 1.0.1
Post by: DaVince on March 29, 2015, 06:29:48 am
Awesome! Good work. Any way to compile this under Linux?
Title: Re: minisphere 1.0.1
Post by: Flying Jester on March 29, 2015, 06:57:41 am
So, I just tried compiling Minisphere on OS X.

How high is the warning level on MSVC turned up for you? You have on bazillion implicit-int and default-return warnings now...it wasn't like that before.

I opened another pull request. I had to remove min/max again, and itoa is nonstandard and does not exist outside MSVC and GCC, but it's just a few lines different, plus the SCons files. The bare minimum to confirm that everything is compiling correctly.
Title: Re: minisphere 1.0.1
Post by: Fat Cerberus on March 29, 2015, 10:00:21 am

So, I just tried compiling Minisphere on OS X.

How high is the warning level on MSVC turned up for you? You have on bazillion implicit-int and default-return warnings now...it wasn't like that before.

I opened another pull request. I had to remove min/max again, and itoa is nonstandard and does not exist outside MSVC and GCC, but it's just a few lines different, plus the SCons files. The bare minimum to confirm that everything is compiling correctly.


The warning level is whatever MSVC's default is.  It doesn't usually warn for implicit conversions, so I don't bother with casts.  I'm trying to keep those to a minimum anyway to make the code easy to read; if given the choice between cluttering up the code with casts or putting up with warnings, I'll generally choose the latter.  If I wanted to have to cast everything I'd be using c++ not C...

Long story short, I find implicit conversion warnings patronizing. :P

A min/max snuck back in?  I've been using fmin/fmax... Must have been a brainfart on my part.

The default returns are actually spurious and can be ignored.  There is no platform-independent way to declare a no-return function so I just put up with them.  duk_error_ni() calls duk_error_va_raw, which calls longjmp and thus doesn't return.  The compiler isn't smart enough to figure that out though, so you get warnings.  Before I was calling the built in duk_error directly, which IS declared no-return, which is why you didn't get them before.
Title: Re: minisphere 1.0.1
Post by: Fat Cerberus on March 29, 2015, 11:38:22 am

Awesome! Good work. Any way to compile this under Linux?


Well as you can see, Jester got it to compile under OS X, so in theory nothing should be stopping it from compiling in Linux either.  I tried to write the code to be as portable as possible, and libraries were also carefully chosen to this end (Allegro, Duktape and Dyad are all supposed to work on at least the Big 3).  Unfortunately Win8.1 doesn't appreciate dual boot configurations and I don't have a spare computer, or I would have tried this myself in Ubuntu.
Title: Re: minisphere 1.0.1
Post by: Flying Jester on March 29, 2015, 05:48:14 pm
Note that it aborts on something about joystick in Allegro when starting up on OS X. But it should still compile with GCC, at least on 64-bit systems.

https://github.com/FlyingJester/TSPR/blob/master/noreturn.h (https://github.com/FlyingJester/TSPR/blob/master/noreturn.h)

It's not too difficult to do noreturn in a way that MSVC, Clang, and GCC accept :)
Title: Re: minisphere 1.0.1
Post by: Fat Cerberus on March 29, 2015, 06:16:08 pm
Do you know what the joystick error is by any chance?  If I knew the exact message I could google it.
Title: Re: minisphere 1.0.1
Post by: Fat Cerberus on March 29, 2015, 10:36:55 pm
@Flying Jester:
Well, I experimented with raising the warning level in MSVC from level 3 to level 4, and got a ton of spurious warnings, so I'm thinking I will probably lower it again.  What irks me most is all the C4706 warnings (assignment within conditional expression) from perfectly idiomatic lines like this:
Code: [Select]
if (!(image = create_subimage(parent, x, y, width, height))) goto on_error;


That code is perfectly clear in its intent, the warning is not needed.  And sure, I could go around disabling warnings individually, but that starts to defeat the purpose of the higher warning level, so I might as well just leave it at level 3 where it compiles cleanly.  I think the low number of major bugs putting together an engine this fast speaks to the fact of me knowing what I'm doing here without the compiler complaining at me. :P

Also: Don't ever try to compile a Win32 app in MSVC with -Wall - You will get a crapton of warnings (as in, tens of thousands! :o) just from the Windows headers alone.
Title: Re: minisphere 1.0.1
Post by: Flying Jester on March 29, 2015, 11:21:03 pm
Some of it a little concerning, even so. This is on the default the settings for GCC/Clang. Clang not pretending to be GCC is even less forgiving.

You should probably at least add safe-guard return statements or use some macros to fix the non-return statements. Technically it's undefined behaviour to even call a function like that.

Some some stuff, like this:

Code: [Select]

static js_BindJoystickButton(duk_context* ctx)


It's not well formed ANSI C to not have a return type. GCC is gracious about it, but Clang doesn't accept it.

In this case:
Code: [Select]

minisphere/main.c:267:13: warning: assigning to 'char *' from 'const char *' discards qualifiers [-Wincompatible-pointer-types-discards-qualifiers]
                file_name = file_name != NULL ? file_name + 1 : file_path;


This is bad. Not respecting constness is undefined behaviour, as well. If you cannot avoid it then you should add a cast.

...anyway...

In some cases it is telling you that you are not following the standard, but the compiler is willing to let it slide, such as in the default-int, const-incorrectness, and non-returns. The compiler could quite validly deny this code even to compile. These are the ones I'm properly worried about, because a standards-compliant compiler may or may not actually compile it.
Title: Re: minisphere 1.0.1
Post by: Fat Cerberus on March 29, 2015, 11:55:49 pm
The functions with no return type I discovered myself during my foray into /W4, those were accidental and I'm not even sure how that happened.  I must have been falling over tired when I wrote something like static js_BindJoystickButton.  I fixed it in my latest commit.

As for const-correctness, from what I understand - if we're being technical about it it's not undefined behavior to cast away constness (C++ has const_cast for a reason), it's only UB if you write to an actual constant.  I'm not saying this isn't a potential source of bugs of course, just want to dispel any misconceptions.  That said, const-correctness is something I do take seriously (if you look at the code, I declare a good number of const pointers myself), but if MSVC doesn't warn me about it, I may not always catch violations.  But even with /W4, I don't get any such warnings, and /Wall is just unfeasible as mentioned above.  Clang must just be REALLY pedantic.

I did fix the no-return functions in the latest commit, I added a noreturn macro so I can now use noreturn in place of a return type.  I wasn't aware GCC would accept the __attributes__ before the function declaration, I thought it had to go after, which would have complicated the macro.  Thanks for the tip. :)

Oh, and regarding that const violation on line 267, I'm thinking that came about due to confusion with strrchr, which is defined, at least in MSVC, as:
Code: [Select]
char* strrchr(const char*, int);


Not sure who had that bright idea, but that's a const correctness violation waiting to happen--and in fact, one did.  Anyway, I fixed it.
Title: Re: minisphere 1.0.1
Post by: Flying Jester on March 30, 2015, 12:28:56 am
That's my biggest peeve about MSVC. Its standard library tends to take the standard a little loosely, and unlike the GCC and Cland standard libraries it doesn't offer warnings when you use nonstandard functions.


As for const-correctness, from what I understand - if we're being technical about it it's not undefined behavior to cast away constness (C++ has const_cast for a reason), it's only UB if you write to an actual constant.  I'm not saying this isn't a potential source of bugs of course, just want to dispel any misconceptions.  That said, const-correctness is something I do take seriously (if you look at the code, I declare a good number of const pointers myself), but if MSVC doesn't warn me about it, I may not always catch violations.  But even with /W4, I don't get any such warnings, and /Wall is just unfeasible as mentioned above.  Clang must just be REALLY pedantic.


Clang doesn't joke around :D That's why I am so happy that TurboSphere compiles with -Wall -Werror with both GCC-compatiblity and pure Clang mode.
Title: Re: minisphere 1.0.2
Post by: Fat Cerberus on March 30, 2015, 02:33:32 am
minisphere 1.0.2 is up, fixing a lockup when using IsKeyPressed in a loop.  Had to use git bisect to find this one, it had me stumped for over a week.
Title: Re: minisphere 812.8.2 now with awesome 3D!
Post by: Fat Cerberus on April 01, 2015, 01:01:36 pm
Hey, check out the new 812.8.2 version of minisphere!  It's flipping awesome! :D
Title: Re: minisphere 812.8.2 now with awesome 3D!
Post by: Radnen on April 01, 2015, 02:10:12 pm

Hey, check out the new 812.8.2 version of minisphere!  It's flipping awesome! :D


3D!? That's amazing! I'll definitely check it out when I get home. But I'll have to be very careful of the hunger pig.
Title: Re: minisphere 812.8.2 now with awesome 3D!
Post by: Fat Cerberus on April 01, 2015, 11:35:19 pm
Sorry guys, I'm going to have to pull 812.8.2 at midnight.  It seems everyone that tested it ended up getting eaten by that hunger-pig at 8:12pm. :'(

...April fools!  There was no 812.8.2 version of minisphere with awesome 3D rendering, I made it up (not that it wasn't entirely obvious anyway, haha). ;D. I did post a real bug fix release today though (1.0.3), in case anyone missed it.
Title: Re: minisphere 1.0.3
Post by: Fat Cerberus on April 02, 2015, 11:30:18 am
Odd, I can't get Kefka's Revenge to run in minisphere.  I get an error loading jewtheblue.bmp, which doesn't make sense since Allegro is supposed to support BMP.

Edit: Figured it out, the offending image is actually a PNG image saved as .bmp (Thank goodness for hex editors!).  Allegro apparently doesn't appreciate the file extension being changed, while Corona doesn't seem to mind.  I'm debating whether I should code in a workaround for this or just leave it alone and make a note in the readme... I'm leaning towards the latter.

Edit2: Aww, looks like KR isn't going to run in minisphere any time soon as it uses the animation API, which I currently have no means of supporting.
Title: Re: minisphere 1.0.3
Post by: Radnen on April 02, 2015, 09:21:46 pm

Edit: Figured it out, the offending image is actually a PNG image saved as .bmp (Thank goodness for hex editors!).  Allegro apparently doesn't appreciate the file extension being changed, while Corona doesn't seem to mind.  I'm debating whether I should code in a workaround for this or just leave it alone and make a note in the readme... I'm leaning towards the latter.


AFAIK corona uses the file contents to verify the data, while a lot of libraries tend to use the stupid file extension. Our Sphere engines for example can load rws files without .rws appended to it. Using the first 4 chars to verify the filetype.
Title: Re: minisphere 1.0.3
Post by: N E O on April 03, 2015, 02:11:34 pm
Looking forward to reading OSX compilation instructions so I can give this a shot on my Mavericks MacBook Air!
Title: Re: minisphere 1.0.3
Post by: DaVince on April 03, 2015, 02:20:41 pm

Odd, I can't get Kefka's Revenge to run in minisphere.  I get an error loading jewtheblue.bmp, which doesn't make sense since Allegro is supposed to support BMP.

Edit: Figured it out, the offending image is actually a PNG image saved as .bmp (Thank goodness for hex editors!).  Allegro apparently doesn't appreciate the file extension being changed, while Corona doesn't seem to mind.  I'm debating whether I should code in a workaround for this or just leave it alone and make a note in the readme... I'm leaning towards the latter.

Edit2: Aww, looks like KR isn't going to run in minisphere any time soon as it uses the animation API, which I currently have no means of supporting.

It uses MNG for only a select few things, like the little dragon morph animation on the title screen. Maybe they could simply be stubbed?
Title: Re: minisphere 1.0.3
Post by: Fat Cerberus on April 03, 2015, 03:26:45 pm

Looking forward to reading OSX compilation instructions so I can give this a shot on my Mavericks MacBook Air!


There's an SCons script in the repo now, courtesy of Jester.  You should probably be able to use that to build it.  Just keep in mind you will likely get some warnings. :)
Title: Re: minisphere 1.0.3
Post by: DaVince on April 03, 2015, 09:41:53 pm
Tried to compile from master:

Code: [Select]
gcc -o minisphere/font.o -c minisphere/font.c
minisphere/font.c: In function 'free_font':
minisphere/font.c:189:2: error: 'for' loop initial declarations are only allowed in C99 or C11 mode
  for (int i = 0; i < font->num_glyphs; ++i) {
  ^
minisphere/font.c:189:2: note: use option -std=c99, -std=gnu99, -std=c11 or -std=gnu11 to compile your code
scons: *** [minisphere/font.o] Error 1
scons: building terminated because of errors.


:(
Title: Re: minisphere 1.0.3
Post by: Fat Cerberus on April 03, 2015, 09:49:49 pm
Oops, that was some of my earlier code, while I was still in JS mode, I accidentally declared the index variable in the for loop itself, which is illegal ANSI C and requires the compiler to be in C99 mode to compile successfully.  It's odd though, that SCons script should be exactly what Jester used to get it to build, I haven't modified it...

I'll have to go through the code and do some more cleanup.
Title: Re: minisphere 1.0.4
Post by: Fat Cerberus on April 04, 2015, 02:36:52 am
DaVince, try compiling again from the latest master.  I fixed the offending line; there don't appear to be any other instances.
Title: Re: minisphere 1.0.4
Post by: DaVince on April 04, 2015, 07:07:00 am
It seems to have compiled all the source files into object files, but then fails at the next step... Am I missing a dependency, perhaps? I checked, but there's no libfmod in the package repository (Ubuntu), so I'm not sure how to proceed...

Code: [Select]
gcc -o minisphere/msphere minisphere/api.o minisphere/bytearray.o minisphere/color.o minisphere/duktape.o minisphere/dyad.o minisphere/file.o minisphere/font.o minisphere/geometry.o minisphere/image.o minisphere/input.o minisphere/logger.o minisphere/lstring.o minisphere/main.o minisphere/map_engine.o minisphere/obsmap.o minisphere/persons.o minisphere/primitives.o minisphere/rawfile.o minisphere/script.o minisphere/sockets.o minisphere/sound.o minisphere/spriteset.o minisphere/surface.o minisphere/tileset.o minisphere/windowstyle.o -lallegro -lallegro_audio -lallegro_acodec -lallegro_color -lallegro_dialog -lallegro_font -lallegro_image -lallegro_primitives -lallegro_ttf -lallegro_main
/usr/bin/ld: minisphere/duktape.o: undefined reference to symbol 'fmod@@GLIBC_2.2.5'
/lib/x86_64-linux-gnu/libm.so.6: error adding symbols: DSO missing from command line
collect2: error: ld returned 1 exit status
scons: *** [minisphere/msphere] Error 1
scons: building terminated because of errors.


Speaking of dependencies, could you please list them in an INSTALL file or something? I noticed I needed liballegro5-dev liballegro-audio5-dev liballegro-image5-dev and installed the other allegro 5 packages just to be sure that I got all appropriate dependencies too, but I'm not sure what else is necessary. All the Allegro libs I am now noticing in the first line in the log up here, I suppose?
Title: Re: minisphere 1.0.4
Post by: Fat Cerberus on April 04, 2015, 10:09:01 am
fmod is a standard C function (floating point modulo).  No idea what's going on there, but it's nothing to do with minisphere itself, that's for sure.

As for dependencies, Allegro 5 should be the only one.  You were right to install the whole shebang there, as I'm pretty sure I use the majority of them.  My only other two dependencies are Duktape and a small networking library called Dyad, both of whose .c files are included in the project.

Sorry for the lack of build documentation, I'm a Windows developer by nature so I'm used to having a nice IDE (read: Visual Studio) take care of everything for me.  I'd much rather write code for my own program than my build tools, thank you. :P
Title: Re: minisphere 1.0.4
Post by: DaVince on April 04, 2015, 05:29:59 pm
After some research, it seems that -lm needs to be added at the end of the gcc linker command. But I'm not sure how to do this with these scons files.

Edit: figured it out. I added a line after line 40 in SConscript with:
Code: [Select]
"m"


And now it's compiled. I like that you're showing a file dialog to ask for the sgm!
Title: Re: minisphere 1.0.4
Post by: DaVince on April 04, 2015, 05:46:27 pm
I tried running Sir Boingers. It ran the main menu just fine (after changing all const keywords to var and resaving some scripts as UTF-8).

There is a bug, however: when I try entering the game, or other parts of it, it errors out. Except since I use the space bar, the window closes almost immediately and I can't read the error message at my leasure. Could you perhaps add a short delay (like half a second) before reading the space bar input, or only start checking for it once the last in-game keypress has been released?

Also, after taking a quick screenshot while the error displayed, it turns out it couldn't load the map. (default/tutorial.rmp). Do you know what's up with that? Is it because it's in a subfolder (maps/default/tutorial.rmp)? And do you think you could output the error to the console too?

Edit: just noticed it plays back the menu sound effects only once, too. Is it because I'm stopping the last instance and then immediately playing it back again, perhaps?
Title: Re: minisphere 1.0.4
Post by: Fat Cerberus on April 04, 2015, 05:53:09 pm

I tried running Sir Boingers. It ran the main menu just fine (after changing all const keywords to var and resaving some scripts as UTF-8).

There is a bug, however: when I try entering the game, or other parts of it, it errors out. Except since I use the space bar, the window closes almost immediately and I can't read the error message at my leasure. Could you perhaps add a short delay (like half a second) before reading the space bar input, or only start checking for it once the last in-game keypress has been released?

Also, after taking a quick screenshot while the error displayed, it turns out it couldn't load the map. (default/tutorial.rmp). Do you know what's up with that? Is it because it's in a subfolder (maps/default/tutorial.rmp)? And do you think you could output the error to the console too?

Edit: just noticed it plays back the menu sound effects only once, too. Is it because I'm stopping the last instance and then immediately playing it back again, perhaps?


Which OS is this?  If anything other than Windows, I'm impressed you got this far, it means I did something right! ;D  Still, I'd like to diagnose some of this stuff.  I'll go get the game in question and see what I can do.

As for the spacebar to close the error, I originally made it so you had to press Escape to close it, but added the spacebar option for convenience.  Maybe I should go back to just [Esc] to close out.
Title: Re: minisphere 1.0.4
Post by: Fat Cerberus on April 04, 2015, 07:02:04 pm

Also, after taking a quick screenshot while the error displayed, it turns out it couldn't load the map. (default/tutorial.rmp). Do you know what's up with that? Is it because it's in a subfolder (maps/default/tutorial.rmp)? And do you think you could output the error to the console too?

Edit: just noticed it plays back the menu sound effects only once, too. Is it because I'm stopping the last instance and then immediately playing it back again, perhaps?


Figured out the map issue after stepping through the map load function with the debugger a few times.  The map itself loads just fine, it's actually the tileset that causes the load failure.  Good call on the subdirectory thing: It turns out the tileset filename stored in the RMP file is relative to the directory containing that map--whereas minisphere assumes it's always relative to <gamedir>/maps.  So yeah, having the map in a subdirectory tripped it up.  Shouldn't be too difficult to fix.  1.0.5, here we come!

As for the sounds, I have noticed a few issues with sound myself, but I haven't been able to pin down the cause yet.  Either I'm doing something wrong or Allegro's audio routines are glitched.  I'm thinking more the former, though... ;)  The sound stuff was actually one of the first things I implemented way back in the beginning of minisphere's development (yes, my priorities are odd, don't judge me! :) ), so the code could probably use another go-over.

Edit: So apparently minisphere doesn't like playing the same sound more than once, despite the explicit al_seek_audio_stream_secs(stream, 0.0); in js_Sound_stop().  Further investigation will be needed.  I didn't notice this before because all my sound-playing in Specs is done through Scenario's built-in playSound scenelet, which loads sounds anew each time they're played.  I never bothered to implement caching for that scenelet...

Edit2: Damn it, best I can tell it's a race condition in Allegro.  This is my working theory: Allegro apparently runs the stream in a separate thread. When you call the stream functions, it queues the operation to be done on the worker thread.  So even though I've queued a rewind, I also call play immediately afterwards.  I'm guessing Allegro processes the play operation before the seek, so it immediately stops again as it's still at the end of the stream.

Edit3: Aaand I was right.  When the stream ends "naturally" (manual starts and stops don't seem to affect it), the stream feeder thread terminates and never gets restarted unless you recreate the stream.  Looks like it was an Allegro bug after all!
Title: Re: minisphere 1.0.4
Post by: Radnen on April 05, 2015, 01:29:07 am
Big problem. I can't run Blockman in it because it has trouble parsing all the code files. I got it to stop spewing syntax errors due to my use of const, but now it's giving me a 'not callable' issue that I cannot resolve.

Basically the minimal use case looks like this:
Code: (javascript) [Select]

function SuperClass() {
this.weight = 5;
}

function SubClass() {
SuperClass.call(this);
}

SubClass.prototype = new SuperClass(); // ERROR here.

var a = new SubClass();
Abort(a instanceof SuperClass);


The above works 100% in minisphere. Then why the error? In Blockman I use the above pattern on hundreds of files and at some point it starts saying not callable on a myriad of functions using this design pattern. This games runs just fine in Sphere 1.5 and SSFML, but it gives me not callable issues on what are seemingly normal function calls.

I do some tests and try to spout out some info:
Code: (javascript) [Select]

Abort(FuncInQuestion); // FuncInQuestion() { /*ecma code*/ }
Abort(typeof FuncInQuestion); // function
Abort(FuncInQuestion()); // TypeError: not callable


This happens because the analyzer thought these functions were somehow set to not callable, when in fact they are. I'm not sure if it's a scoping thing either because if it were, the first two tests would fail since the function in question could not be found. So this is really weird indeed.
Title: Re: minisphere 1.0.4
Post by: Fat Cerberus on April 05, 2015, 01:33:40 am
Odd, sounds like a Duktape bug.  I could post an issue on the Duktape repo, but I'd need a use case that can reproduce it reliably.  But from the sounds of things I'm guessing you can't reproduce it on command...?
Title: Re: minisphere 1.0.4
Post by: Radnen on April 05, 2015, 01:59:08 am
It's hard when it's a random function call in a many thousand line project. :/

Sometimes it's not just that pattern above but as simple as this:
Code: (javascript) [Select]

function Item() {
    this.inner_item = new Item2(); // not callable error
}


I mean, the Item2() in the example above had been loaded, I can print various qualities about it, but I just can't call it.
Title: Re: minisphere 1.0.4
Post by: Fat Cerberus on April 05, 2015, 02:10:35 am
Code: (javascript) [Select]
var Item = function() {
        return function Item() {
                this.weight = 5;
        }
}

var a = new Item();
Abort(a.weight); // undefined


No bug, no ES deficiency, this code is just broken:
Code: (javascript) [Select]
Abort(a); // function Item() {/* ecmascript */}


[forrestgump]And that's all I have to say about that.[/forrestgump] :P
Title: Re: minisphere 1.0.4
Post by: Radnen on April 05, 2015, 02:16:20 am
No, I'm mentally retarded there. But I'm still getting a not callable issue. Don't know what it could be tho.

It should be:
Code: (javascript) [Select]
var Item = (function() {
        return function Item() {
                this.weight = 5;
        }
})();

var a = new Item();
Abort(a.weight); // 5

Title: Re: minisphere 1.0.4
Post by: Fat Cerberus on April 05, 2015, 03:03:41 am
By the way: minisphere includes an Alert() function that just throws up a message box without terminating the game.  Might be useful for these little testcases.

edit: also where can I get the latest Blockman?  I have an older copy right now from the Spherical Downloads drive, which is actually one of the games I used for compatibility testing. :-). But that version seems to run just fine, so I'm unable to reproduce this issue with it.
Title: Re: minisphere 1.0.4
Post by: Fat Cerberus on April 05, 2015, 03:18:17 am
Oh, and by the way, your whodunnit game works flawlessly since 1.0.1. ;D
Title: Re: minisphere 1.0.4
Post by: Radnen on April 05, 2015, 04:22:12 am
I'll send you a D/L link in a PM since I still don't want this game to be released publicly yet.
Title: Re: minisphere 1.0.4
Post by: Fat Cerberus on April 05, 2015, 09:56:22 am
Interesting, I stepped through the internal call code inside Duktape for the offending call, and it successfully gets into the bytecode for constructor.  So this "not callable" error is very baffling, since it actually does call the function...

Dammit, I just figured out: the error is misleading.  I have an error callback that modifies the filename and line number for error messages starting with "not " because otherwise Duktape doesn't report that info for invalid parameters.  The callback uses the info from one step up the call stack, which means the error is actually inside the constructor.  I will keep investigating.

Edit: Yep, I sent us both on a wild goose chase.  I fixed the error callback to pass through "not callable" errors and here's the actual offending line of code:
Code: (javascript) [Select]
this.__defineGetter__("focusedControl", function() { return _focused_control; });


.......... *facepalm*

I'm going to go sheepishly add __defineGetter__/__defineSetter__ polyfills now... :-[
Title: Re: minisphere 1.0.4
Post by: Fat Cerberus on April 05, 2015, 10:49:29 am
Well, several fixes (and const removals) later I got it to get to the main menu, but I can't move the cursor or even confirm my selection.  Apparently something's borked with my input routines... which is odd, since input seems to work fine in every other game I've tested.
Title: Re: minisphere 1.0.4
Post by: DaVince on April 05, 2015, 11:46:26 am
I'm on Linux, so nice work getting it to work this far! :)


As for the spacebar to close the error, I originally made it so you had to press Escape to close it, but added the spacebar option for convenience.  Maybe I should go back to just [Esc] to close out.

Nah, then the same problem would occur when the error message displays during an esc key press action. I like this short pause you added better.

Still errors out though (http://i.imgur.com/269oLjG.png). Compiled from latest source (updated "an hour ago").
Title: Re: minisphere 1.0.4
Post by: Fat Cerberus on April 05, 2015, 11:50:06 am
Yeah, I didn't fix the map loading issue yet.  I got sidetracked trying to get Radnen's game running. :P. I'll do the fix later today, it shouldn't take too long.  Just have to extract the directory info from the map path and prepend that to the tileset name and I should be golden.
Title: Re: minisphere 1.0.4
Post by: DaVince on April 05, 2015, 11:54:53 am
Ah, okay! Sorry to be premature, I saw updates and just assumed. :P In the meantime, I found another crash (http://i.imgur.com/uV04vlt.png) when trying to open either option 2 or 3 in the menu. I'm not sure what's going on because the offending line looks rather innocent:

Code: [Select]
for (var y = 0; y < this.row_amount; y++)
    { for (var x = 0; x < this.col_amount; x++)
      { var opt = this.options[x][y]; //<<<<< it errors on this line
        if (opt != undefined)
Title: Re: minisphere 1.0.4
Post by: Radnen on April 05, 2015, 12:17:49 pm

Well, several fixes (and const removals) later I got it to get to the main menu, but I can't move the cursor or even confirm my selection.  Apparently something's borked with my input routines... which is odd, since input seems to work fine in every other game I've tested.


SSFML uses a slightly different keymapping. Rename Options2.ini to Options.ini and it'll be set to use the keymaps for Sphere 1.5. I really need to manually change the keymaps, I wonder, do you use 1.5's keymaps?
Title: Re: minisphere 1.0.4
Post by: Fat Cerberus on April 05, 2015, 12:23:45 pm
If you mean GetPlayerKey:

Menu: Tab (I think)
ABXY: Z, X, C, V
Title: Re: minisphere 1.0.4
Post by: DaVince on April 05, 2015, 12:58:25 pm
I also encountered the issue of input not being read in one of my old games. The one attached, to be exact. It expects input from the Ctrl key, but nothing happens. Escape does work for some reason.

(Note: give the game 2 seconds to really start. This is on purpose.)
Title: Re: minisphere 1.0.4
Post by: DaVince on April 05, 2015, 01:29:49 pm
I sent Minisphere 1.0.4 to a friend and she sent the following error message back:

(http://i.imgur.com/zjX9YU2.jpg)

Note that I renamed the file to Sarah.exe, put the game in startup, and the only other folder was system. It worked on Wine so I wouldn't assume it wouldn't even run on her system. I have no clue what this error means at all either.
Title: Re: minisphere 1.0.4
Post by: Fat Cerberus on April 05, 2015, 01:33:43 pm
Looks like a function used by allegro doesn't exist in her version of Windows (kernel32 is the OS kernel)...
Title: Re: minisphere 1.0.4
Post by: DaVince on April 05, 2015, 01:49:08 pm
Okay, well, that's Vista. Guess we got that cleared up. Is it possible to show a more informative error?
Title: Re: minisphere 1.0.4
Post by: Fat Cerberus on April 05, 2015, 01:59:03 pm
Not really, that's an OS-triggered error while loading the executable, no way to catch it because the engine isn't even running yet.
Title: Re: minisphere 1.0.4
Post by: DaVince on April 05, 2015, 02:26:06 pm
After a bit of looking around it seems that compiling with an older MSVC or with MingW would solve this. I might mess around with this later (no promises though :P).
Title: Re: minisphere 1.0.4
Post by: Fat Cerberus on April 05, 2015, 02:50:40 pm
I found this blog post:
http://blogs.msdn.com/b/vcblog/archive/2009/08/27/windows-sdk-v7-0-v7-0a-incompatibility-workaround.aspx

A bunch of Win32 APIs were apparently shuffled around in Win7.  It looks like the fix is as simple as defining a preprocessor macro.  That's a simple enough thing to do, I'll add this to the queue for 1.0.5.  The next release is getting a ton of fixes! :D
Title: Re: minisphere 1.0.4
Post by: DaVince on April 05, 2015, 03:26:23 pm
Awesome! Nice work. :D
Title: Re: minisphere 1.0.4
Post by: Fat Cerberus on April 05, 2015, 08:33:21 pm
Alright, 1.0.5 should relax the Win7+ requirement.  It was the MSVC 11 build of Allegro I was using.  I linked minisphere against the MSVC 9 libs and K32GetModuleFileNameExA disappeared from Dependency Walker. :)

I also fixed the maps in subdirectories issue, but now I'm getting an "invalid base value" error whose cause I can't pin down.  It can never be simple, can it...

@Radnen: OOH, I see what you mean about the mappings now.  I opened the options file, and saw this line:
Code: (javascript) [Select]
keys={"upKey":77,"downKey":79,"leftKey":80,"rightKey":78,"AKey":59,"BKey":55,"XKey":60,"YKey":68,"LKey":72,"RKey":76,"startKey":70,"selectKey":61}


...and suddenly understood jack everything. :P  See, minisphere uses Allegro's key constants directly, it doesn't remap them.  Obviously those values are different than whatever Sphere 1.5 defines them as.  I never thought it would be an issue since I expected games to always use the named constants, but didn't account for the possibility of keys being saved out to a file...
Title: Re: minisphere 1.0.4
Post by: Flying Jester on April 05, 2015, 09:23:23 pm
Out of curiosity, does Duktape support ArrayBuffers or 'let'?
Title: Re: minisphere 1.0.4
Post by: Fat Cerberus on April 05, 2015, 09:25:31 pm
I don't believe so.  I know the developer is currently working on implementing TypedArrays though...

`let` is ES6, yes?  It has some ES6 stuff, but I don't believe that's one of them.  I'd have to double-check though.
Title: Re: minisphere 1.0.4
Post by: Radnen on April 05, 2015, 09:36:06 pm

I never thought it would be an issue since I expected games to always use the named constants, but didn't account for the possibility of keys being saved out to a file...


Yeah, the game has the ability for you to set your own key mappings.

I added the ability for the game to recognize default keys on startup. If no options.ini is available it'll fill it out with the correct keys on first run. If you switch sphere engine, just delete options.ini and it'll re-scan the keys again.
Title: Re: minisphere 1.0.4
Post by: Fat Cerberus on April 05, 2015, 11:59:27 pm
Okay, that did the trick, now I can select stuff from the menu.  Unfortunately, now I'm getting the same error as in DaVince's game: Invalid base value, with no filename or line number to indicate where the issue is.  What fun...
Title: Re: minisphere 1.0.4
Post by: Fat Cerberus on April 06, 2015, 12:34:11 am
Radnen, you'll be happy to know I just got Blockman to run in minisphere. ;D  There do appear to be a few glitches, though--such as persons occasionally "jumping" from one place to another during cutscenes instead of moving smoothly, and some camera jerkiness.  Not sure what's causing that.

However, there's unfortunately one thing that prevents it from being fully playable: FollowPerson is not implemented.  I've been putting that off forever... I knew it would come back to bite me eventually. :(
Title: Re: minisphere 1.0.4
Post by: Radnen on April 06, 2015, 12:38:36 am

Radnen, you'll be happy to know I just got Blockman to run in minisphere. ;D  There do appear to be a few glitches, though--such as persons occasionally "jumping" from one place to another during cutscenes instead of moving smoothly, and some camera jerkiness.  Not sure what's causing that.


Don't worry, happens in Sphere too! :P


However, there's unfortunately one thing that prevents it from being fully playable: FollowPerson is not implemented.  I've been putting that off forever... I knew it would come back to bite me eventually. :(


Yeah, I've been wanting to write my own FollowPerson rather than use Sphere's anyways. :P So once you do fix it, I might not use it. There is only 1 instance in the game FollowPerson is used, and it's optional to follow that person.
Title: Re: minisphere 1.0.4
Post by: Fat Cerberus on April 06, 2015, 01:12:46 am


Radnen, you'll be happy to know I just got Blockman to run in minisphere. ;D  There do appear to be a few glitches, though--such as persons occasionally "jumping" from one place to another during cutscenes instead of moving smoothly, and some camera jerkiness.  Not sure what's causing that.


Don't worry, happens in Sphere too! :P


Oh good, so nothing to fix on my end.  Phew!
Title: Re: minisphere 1.0.5
Post by: Fat Cerberus on April 06, 2015, 02:22:26 am
Well, that was a hell of a ride. 1.0.5 is done.  This release represents a huge chunk of maintenance and fixes about 7 different bugs.  Most of them were pretty minor in the grand scheme of things, but they were enough to prevent a lot of games from running, so fixing them is important.

@DaVince: Could you have your friend try minisphere on Vista again, this time with 1.0.5?  I'm pretty sure the Win7+ requirement is gone, but I don't currently have access to a Vista or XP machine to know for sure.  Also: Sir Boingers now (mostly) works. It tends to error out on death/damage though, due to the stricter type checking in minisphere (this is mentioned in the readme).  With a few edits to the game itself, it should now run flawlessly.  Awesome game, by the way. :D
Title: Re: minisphere 1.0.5
Post by: Radnen on April 06, 2015, 02:25:50 am

Well, that was a hell of a ride. 1.0.5 is done.  This release represents a huge chunk of maintenance and fixes about 7 different bugs.  Most of them were pretty minor in the grand scheme of things, but they were enough to prevent a lot of games from running, so fixing them is important.


I know exactly what you mean by that. ;)
Title: Re: minisphere 1.0.5
Post by: DaVince on April 06, 2015, 09:46:44 am

@DaVince: Could you have your friend try minisphere on Vista again, this time with 1.0.5?  I'm pretty sure the Win7+ requirement is gone, but I don't currently have access to a Vista or XP machine to know for sure.  Also: Sir Boingers now (mostly) works. It tends to error out on death/damage though, due to the stricter type checking in minisphere (this is mentioned in the readme).  With a few edits to the game itself, it should now run flawlessly.  Awesome game, by the way. :D

Sure, I'll send it to her and let you know. And thanks! :)

I've also fixed those bugs, though I'll hold off on releasing a Sir Boingers with such a minor modification until I've done more tests/fixing... and I might as well continue actual development on the game now that I have the opportunity.

Also, I couldn't help but notice that the error displayed was simply "not boolean" - it didn't tell me what variable exactly it was having trouble with. While that was no issue whatsoever for my simple code, imagine it happening on a more complex line.
Title: Re: minisphere 1.0.5
Post by: Fat Cerberus on April 06, 2015, 10:59:33 am

Also, I couldn't help but notice that the error displayed was simply "not boolean" - it didn't tell me what variable exactly it was having trouble with. While that was no issue whatsoever for my simple code, imagine it happening on a more complex line.


A bit of background on that.  Here's what an API function looks like internally:
Code: (javascript) [Select]
static duk_ret_t
js_SetPersonVisible(duk_context* ctx)
{
const char* name = duk_require_string(ctx, 0);
bool is_visible = duk_require_boolean(ctx, 1);

person_t* person;

if ((person = find_person(name)) == NULL)
duk_error_ni(ctx, -1, DUK_ERR_REFERENCE_ERROR, "SetPersonVisible(): Person '%s' doesn't exist", name);
person->is_visible = is_visible;
return 0;
}


duk_require_*() is shorthand for "get the value at this location on stack, but if it's not the type I'm looking for, throw an exception".  It's very convenient, and keeps the code size down.  But normally you don't even get a filename and line number for such errors: the only reason you do in minisphere is because I manually pull it from the call stack. Otherwise you'd just get a completely useless bare "not boolean" error.

I could fix it, of course, but that would require a bunch of extra conditional checks in every single API function, which really puts me off.  Go look at the Sphere source sometime: For a lot of API calls, half the function is argument checking (which, mind you, doesn't really make sense since it still ends up type-coercing 90% of what you throw at it anyway; Spidermonkey is a ridiculously verbose API), and I'd prefer to avoid that here.  The "mini" in minisphere is really about the code size more than anything; the fact that the engine itself takes up less space than Sphere 1.5 is just icing on the cake.
Title: Re: minisphere 1.0.5
Post by: DaVince on April 06, 2015, 11:19:24 am
Okay. If it can't be helped, it can't be helped. Thanks for the explanation.

I really love the fact that all you really need to distribute a game is the engine binary, startup and system folder, by the way. So much cleaner.
Title: Re: minisphere 1.0.5
Post by: Fat Cerberus on April 06, 2015, 11:29:00 am

I really love the fact that all you really need to distribute a game is the engine binary, startup and system folder, by the way. So much cleaner.


Yep, early in development you also needed the Allegro monolith DLL, but a static link fixed that handily (I static link to the MSVC runtime for the same reason).  I always hated that Sphere requires you to distribute about 10 different DLL files along with the engine.  Worse, some of those are only needed for the editor, and it's not necessarily obvious which ones.  So you have to basically guess which ones you can leave out, or just copy them all and bloat your distributable unnecessarily.  Having everything self-contained in the engine executable means that's no longer an issue. :)

If I could, I would also get rid of the system folder requirement, but as a usable system font--at the very least--is required for the engine to even run, not much I can do there.
Title: Re: minisphere 1.0.6
Post by: Fat Cerberus on April 06, 2015, 12:31:25 pm
Just posted a quick hotfix release, 1.0.6, fixing a map collision bug that rendered Sir Boingers unwinnable.
Title: Re: minisphere 1.0.6
Post by: DaVince on April 06, 2015, 01:37:39 pm
Could you insert this after line 40 in minisphere/SConscript? It fixes compilation on Linux:

Code: [Select]
    "m",


Well, I suppose that's not supposed to be in a variable called allegro_libs, but hell if I know where else you're supposed to put it. :P
Title: Re: minisphere 1.0.6
Post by: Fat Cerberus on April 06, 2015, 02:24:30 pm
Done. :)
Title: Re: minisphere 1.0.6
Post by: Fat Cerberus on April 06, 2015, 03:34:55 pm
@Radnen: Curse you and your evil ways! :P  Apparently you delete persons in a command generator script?  minisphere doesn't particularly appreciate entities being pulled out from under it, and it leads to all sorts of random crashes.  Should be fun to fix...
Title: Re: minisphere 1.0.6
Post by: Radnen on April 06, 2015, 07:34:42 pm

@Radnen: Curse you and your evil ways! :P  Apparently you delete persons in a command generator script?  minisphere doesn't particularly appreciate entities being pulled out from under it, and it leads to all sorts of random crashes.  Should be fun to fix...


I guess, I don't remember. I try not to do that in a command generator. In fact I never aim to use it because in Sphere1.5 it's really slow performing.

A pattern I do use is this:

1. I set person to a death animation.
2. I queue animation commands.
3. I queue a person script that destroys the person.

That way the death script is cleanly executed when the animation ends. Otherwise I'm set using an update script, continuously checking for a final state and then destroying the person, making sure I don't accidentally destroy an already destroyed person (which ought to just do nothing, come to think of it).
Title: Re: minisphere 1.0.6
Post by: Fat Cerberus on April 06, 2015, 07:40:30 pm
That's what's crashing it, the person script destroys the entity but the engine tries to read the next command and crashes because the underlying person structure has already been freed.  I didn't anticipate this eventuality, so minisphere doesn't handle it very well.
Title: Re: minisphere 1.0.6
Post by: Radnen on April 06, 2015, 07:49:13 pm
Oh, I see what you mean by command generator, I use the person command queue. I've never been a fan of the GENERATE_COMMANDS command, I'm not sure what it does and SSFML does not have that implemented. I say I'm not sure because it slows down a lot when you put logic in it, like random movement. I found that an update script that iterates over the entities on the map is a 1000 times faster than using the GENERATE COMMANDS script area.

So here is my feature request: Does minisphere's executable icon change? I'm not sure how that works, but is there a way you can set up a build script such that you can build minisphere with your own icon? It's not something I imagine most people doing, but it could be great.

I found minisphere to be way easier than Sphere 1.6 to build, less dependencies and less odd code to go wrong.
Title: Re: minisphere 1.0.6
Post by: Flying Jester on April 06, 2015, 08:22:41 pm
We should, at some point, try to document what all these commands really do. Some are self explanatory, but others like GENERATE_COMMANDS don't really make sense to me. Even COMMAND_WAIT doesn't seem clear to me.
Title: Re: minisphere 1.0.6
Post by: Fat Cerberus on April 06, 2015, 08:27:47 pm

We should, at some point, try to document what all these commands really do. Some are self explanatory, but others like GENERATE_COMMANDS don't really make sense to me. Even COMMAND_WAIT doesn't seem clear to me.


COMMAND_WAIT you insert into the command queue if you want the entity to not do anything for one frame, but otherwise still be under programmatic control.  As for the command generator, the engine calls it for a person on any frame that there is nothing else in that person's queue. Youre supposed to use it to implement automatic movement, e.g. for NPCs.  As for why it's so slow, I have a theory that sphere recompiles it every time it's called.  It should be fast in minisphere since I pre-compile all person scripts.
Title: Re: minisphere 1.0.6
Post by: DaVince on April 06, 2015, 08:32:33 pm
ON_GENERATE_COMMANDS is used to run commands on each frame. (I used it to make people walk and look around in Pokemon GS/SS.)
COMMAND_WAIT is used to add a short pause where a character does nothing for a frame. Useful when you're inputting a list of commands and want to run it all (like walk north, wait a bit, then walk back south). I used that one in GS/SS too.

I wasn't exactly *good* back when I worked on that game, of course. :P But I think COMMAND_WAIT has its uses. ON_GENERATE_COMMANDS, not so much.

...Ninja'd.
Title: Re: minisphere 1.0.6
Post by: Fat Cerberus on April 06, 2015, 09:06:20 pm
The perks of making a compatible engine: you learn how the original works inside and out. :P  up to and including all the annoying little idiosynchrosies that I nonetheless have to emulate to avoid breaking things.  I'm not shy about fixing outright bugs though.  For that reason minisphere will probably never be 100% compatible with Sphere, but that's fine.  If at the end of the day I'm only at 90% parity with vanilla due to bug fixes, I'll take it.

Of course it probably helps that I have a knack for picking up on little subtleties.  Great example being, it didn't take me long at all to figure out that Sphere doesn't destroy entities created before a MapEngine() call, which I believe bit Radnen during SSFML development.  And that even made sense to me: the flag is called `destroy_with_map`, and no map is destroyed in a map engine start since none is loaded yet.

Honestly I think I've had more fun with this project than if I just made a whole new, unrelated engine, which was my original plan.  Getting to see how Sphere works under the hood was worth it all by itself. :)

As for command generator scripts, I wouldn't give them up for anything.  It helps keep each entity self-contained without having to feed NPCs movement commands from somewhere else in the code (the update script, for instance).  Any time the engine offers to manage something for me, I'll jump at the chance.  Less bookkeeping to bloat my own code that way. :P
Title: Re: minisphere 1.0.6
Post by: Fat Cerberus on April 07, 2015, 02:10:25 am
I just added the Allegro libs to the repo.  This makes it ridiculously simple to build the engine on Windows as everything needed is now included--you just need to provide MSVC, which is also simple thanks to VS Community.  One of the many benefits of having all your dependencies be MIT/zlib licensed--you can just throw the files in and not have to worry about licensing issues. :)

Obviously there are no issues on Linux or OS X thanks to the SCons script, but I know compiling in Windows can be hell (and I develop in Windows, so that's saying something! :P), so doing this will make everyone's life a lot easier.
Title: Re: minisphere 1.0.6
Post by: Radnen on April 07, 2015, 02:23:12 am
There is no NuGet for C projects? If so, I guess Allegro isn't there... I just took dependencies out of Sphere Studio and into NuGet packages to decrease the filesize of the git repo. Too bad though since it's been logged in the commit history, so those old DLL files will truly stay in it forever. :/
Title: Re: minisphere 1.0.6
Post by: Fat Cerberus on April 07, 2015, 02:27:28 am
No Allegro on NuGet, I've already checked.  That was the first thing I tried, but no such luck.  No DLLs in mine though, just about 13MB worth of .lib files... Oh well.  Note that the debug build is static-linked now as well.  Maybe I'll think about adding console logging to it now...

Edit: Wait, no, make that 5MB.  Not too bad, then.  Not sure where I got 13 from...
Title: Re: minisphere 1.0.6
Post by: DaVince on April 07, 2015, 06:21:53 am

Of course it probably helps that I have a knack for picking up on little subtleties.  Great example being, it didn't take me long at all to figure out that Sphere doesn't destroy entities created before a MapEngine() call, which I believe bit Radnen during SSFML development.  And that even made sense to me: the flag is called `destroy_with_map`, and no map is destroyed in a map engine start since none is loaded yet.


Actually, it seems like there's a bug regarding not destroying persons: when Sir Boingers switches to the next map, all the entities from the previous map are spawned at the entry point location. Which means I get to see this when starting level 2. Those objects (the flame, blue pad and spike ball above it) are all from level 1.

(http://i.imgur.com/QDutsU8.png)

This causes "person does not exist" issues down the line (like when dying/restarting the level).

Edit: by the way, you can experience this quickly for yourself by pressing F7 (the skip level key). It accumulates with each level.

Edit 2: mp3 is not supported, the engine crashes with a playback error. Is that right?
Title: Re: minisphere 1.0.6
Post by: DaVince on April 07, 2015, 07:38:26 am
Okay, I've found out what the "Invalid base value" error was. I suspect that my code had a bug in it - namely that it would try to access an array inside an array, except the outer array would sometimes be undefined. What I mean is that there's a possibility that x in this.options[ x ][ y ] might have been undefined. Original Sphere seemed to be able to handle this.options[undefined][0] but Duktape doesn't. (And really, why should it? :) )

After adding this line to my code, everything is fine again:
Code: [Select]
if (this.options[x] == undefined) continue;
Title: Re: minisphere 1.0.6
Post by: Fat Cerberus on April 07, 2015, 09:24:40 am
Is it bad that every time someone reports a major bug I rub my hands together excitedly in a "Let's get to work!" kind of way? :P

First, the simple one: That array thing is quite odd.  Here's a little test case:
Code: (javascript) [Select]
var hippo = [ [ "maggie" ] ];
var x, y = 0;
Abort(hippo[x][y]);


This fails in both minisphere and vanilla.  The errors are:
minisphere - TypeError: "invalid base value"
Sphere 1.5 - TypeError: "hippo[ x ] has no properties"

So, not sure what's going on, maybe behavior is different enough between the two engines that x ends up undefined sometimes in minisphere while it never does in Sphere.  I don't know.

As for the persons respawning at the start of the next level, I noticed that myself, but thanks to a lack of testing in Sphere 1.5 I didn't realize it was a minisphere bug--I thought it was a bug in Sir Boingers.  Are you exiting and restarting the map engine when switching levels by any chance?  Because I'm thinking that's what it is--minisphere normally removes transient persons when switching maps just like in Sphere, but this doesn't happen if you stop and restart the map engine.  Apparently it does in Sphere.

Oh and yes, no mp3 support sadly :(.  Allegro apparently doesn't include it, I assume due to licensing issues, and I haven't yet found any addons to re-add support for it.
Title: Re: minisphere 1.0.6
Post by: DaVince on April 07, 2015, 09:30:27 am
Yep, I close and reopen the map engine. It ended up working out better (the map engine only runs when necessary - I can finish a map, run some code, then continue with the next one in a more convenient way). That must be the issue, then.

And my test case doesn't work in regular Sphere either (errors out with "has no properties" too). I'm not exactly sure why my fix worked and the game works fine in regular Sphere.
Title: Re: minisphere 1.0.6
Post by: Fat Cerberus on April 07, 2015, 11:02:27 am
I fixed the bug.  Try the latest master.  :D
Title: Re: minisphere 1.0.6
Post by: casiotone on April 07, 2015, 12:14:10 pm

Is it bad that every time someone reports a major bug I rub my hands together excitedly in a "Let's get to work!" kind of way? :P

First, the simple one: That array thing is quite odd.  Here's a little test case:
Code: (javascript) [Select]
var hippo = [ [ "maggie" ] ];
var x, y = 0;
Abort(hippo[x][y]);


This fails in both minisphere and vanilla.  The errors are:
minisphere - TypeError: "invalid base value"
Sphere 1.5 - TypeError: "hippo[ x ] has no properties"

So, not sure what's going on, maybe behavior is different enough between the two engines that x ends up undefined sometimes in minisphere while it never does in Sphere.  I don't know.


Sounds like duktape doesn't like using `undefined` as an object key. This is probably a bug, it should coerce to a string first, which SM and V8 do. e.g. hippo[undefined] = true; should be equivalent to hippo['undefined'] = true;.

Just wondering, what happens if you try:

Code: [Select]

var array = [];
array[{}] = true;
Abort(Object.keys(array));


This should give you "[object Object]" as the only key in the object.
Title: Re: minisphere 1.0.6
Post by: Fat Cerberus on April 07, 2015, 12:44:38 pm

Sounds like duktape doesn't like using `undefined` as an object key. This is probably a bug, it should coerce to a string first, which SM and V8 do. e.g. hippo[undefined] = true; should be equivalent to hippo['undefined'] = true;.

Just wondering, what happens if you try:

Code: [Select]

var array = [];
array[{}] = true;
Abort(Object.keys(array));


This should give you "[object Object]" as the only key in the object.


It does.  Also, changing array[{}] to array[undefined] yields 'undefined' as the final output.
Title: Re: minisphere 1.0.6
Post by: Defiant on April 07, 2015, 08:22:04 pm
So I downloaded miniSphere the other day, and maybe I'm a little dunce, but does this work on XP? I keep getting an error saying it's not a vaild Win32 application, is there something I'm missing?
Title: Re: minisphere 1.0.6
Post by: Fat Cerberus on April 07, 2015, 08:35:51 pm
That doesn't make sense.  I could see if maybe it wasn't compatible with XP and crashed or something, but "not a valid win32 application" suggests the .exe got corrupted somehow.  It's not a 64-bit build either... Odd.
Title: Re: minisphere 1.0.6
Post by: Defiant on April 07, 2015, 08:53:41 pm
I've tried downloading from both Git-hub and the google drive links, both throw that error.
Title: Re: minisphere 1.0.6
Post by: Fat Cerberus on April 07, 2015, 08:56:41 pm
From GitHub... You compiled it yourself?  It definitely shouldn't be showing that particular error.  The only time you should get that is if the .exe is either corrupt or 64-bit, and since I know for a fact it's not the latter...

I will do some research myself.
Title: Re: minisphere 1.0.6
Post by: Fat Cerberus on April 07, 2015, 09:02:32 pm
Okay, just found out I have to change the MSVC project configuration to get it to run on XP.  Apparently MSVC 2012 and later only target Vista+ by default.  My bad.

That doesn't explain the error if you're building it yourself though... Sounds like a virus.
Title: Re: minisphere 1.0.6
Post by: Defiant on April 07, 2015, 09:05:46 pm
Brain fart, I thought there was a link on Github for the pre-compiled version of it, I remember downloading the source, but never tried to compile it, my bad. So yea, only actually downloaded it from the Google drive on a couple separate instances and same thing.
Title: Re: minisphere 1.0.6
Post by: Fat Cerberus on April 07, 2015, 09:11:22 pm
Alright, then that's my fault.  A quick toggle in MSVC will fix it, chalk up another one for 1.0.7. :P. Thanks for reporting. :D
Title: Re: minisphere 1.0.6
Post by: Fat Cerberus on April 07, 2015, 10:32:11 pm
Okay, everybody, word to the wise for Sphere implementors: Don't use a texture atlas for windowstyles.  For a while now I've been trying to figure out what happened to cause the main screen in Radnen's SSFML test suite to slow down to a crawl (~20 fps) when it ran well before.  When I first noticed this, I ran minisphere in the profiler (VS 2013 Community is awesome) and found out Allegro was switching to software rendering when drawing the windowstyle.  That was about a week ago, and then I forgot about it because I had no idea what might be causing it.

Just now it finally dawned on me: Quite a while ago I modified the windowstyle loader to load the bitmaps into an atlas instead of individually.  The reason for the switch to software rendering is because I use U/V tiling... which can't be done in hardware for only part of a texture, so Allegro switches to software to pull it off.

Of course, the flipside of this illustrates just why I love Allegro: It does everything in its power to service a request without me having to write a ton of fallback code.  This keeps minisphere's codebase small and still ensures it will run on as many systems as possible. 8)
Title: Re: minisphere 1.0.6
Post by: Radnen on April 08, 2015, 12:16:46 am

Oh and yes, no mp3 support sadly :(.  Allegro apparently doesn't include it, I assume due to licensing issues, and I haven't yet found any addons to re-add support for it.


SSFML can play mp3, but no idea on the licensing.

The Sphere Studio can, but it's an IrrKlang plugin, so I can remove the support at any time if licensing ever becomes an issue.

I thought allegro uses any audio codecs installed/recognized by it. It's odd for it to not have that ability, unless I'm missing something.
Title: Re: minisphere 1.0.6
Post by: Fat Cerberus on April 08, 2015, 12:19:51 am
MP3 is patented and requires a licensing fee and/or royalties if your project is large enough.  Hence most open-source libraries leave it out.  That's why Vorbis was invented. :P
Title: Re: minisphere 1.0.6
Post by: casiotone on April 08, 2015, 03:56:23 am


Sounds like duktape doesn't like using `undefined` as an object key. This is probably a bug, it should coerce to a string first, which SM and V8 do. e.g. hippo[undefined] = true; should be equivalent to hippo['undefined'] = true;.

Just wondering, what happens if you try:

Code: [Select]

var array = [];
array[{}] = true;
Abort(Object.keys(array));


This should give you "[object Object]" as the only key in the object.


It does.  Also, changing array[{}] to array[undefined] yields 'undefined' as the final output.

Well, that *is* weird then.
Title: Re: minisphere 1.0.6
Post by: Fat Cerberus on April 08, 2015, 03:01:38 pm
minisphere 1.0.7 is coming.  I'm almost tempted to do a version bump to 1.1 due to both Duktape and Allegro receiving major upgrades (1.2.1 and 5.1.9 respectively), but as that's under-the-hood, I think I will keep it at 1.0 for now.

In any case, the Allegro upgrade shrunk the size of the engine from around 2.5mb down to about 2mb--and even seems to have made things faster! ;D
Title: Re: minisphere 1.0.7
Post by: Fat Cerberus on April 09, 2015, 02:28:24 pm
1.0.7 is out!

Radnen, try running Blockman in minisphere now.  It works much better with the fixes in this release.  Also: The engine is now under 2MB and still completely self-contained!

@Defiant: Could you try this latest release on XP?  I ended up having to build Allegro and all its dependencies myself with VS2013 in XP mode (9 static libraries for those keeping count  :-\), so it better work now or I might have to murder someone. :P
Title: Re: minisphere 1.0.7
Post by: Fat Cerberus on April 10, 2015, 02:06:04 am
This just in: I got the minisphere engine binary down to under 1.5MB without sacrificing any functionality. :)  Compiling Allegro myself was totally worth it, I can leave out all the useless stuff I don't need (like OpenAL, which Allegro doesn't even really take advantage of).  This means the engine is less than a third the size of Sphere 1.5! (By my count, 6.5MB, including the config utility and video plugins and excluding the editor)

minisphere truly lives up to its name! 8)

Also fixed: The audio deadlocks.  I may not have mentioned it, but after the Allegro upgrade I started seeing random hangs when playing sounds.  I'm guessing these were caused by OpenAL, since that's the biggest thing I excised from Allegro and the lockups have since stopped happening. *crosses fingers*
Title: Re: minisphere 1.0.8
Post by: Defiant on April 10, 2015, 07:47:58 pm
@Lord English:
It's at least a valid application now, but it crashes on opening it.

Title: Re: minisphere 1.0.8
Post by: Fat Cerberus on April 10, 2015, 07:52:10 pm
Hm, I might have to set up a virtual machine with XP for testing.  Not sure why it's crashing like that, if you click for more details (the Click Here link), what does it say?
Title: Re: minisphere 1.0.8
Post by: Defiant on April 10, 2015, 08:00:19 pm
It comes up with the screenshot below... but when I click technical details I can't copy&paste the information there, and it's got all the modules and some hex data.
Title: Re: minisphere 1.0.8
Post by: Fat Cerberus on April 10, 2015, 08:10:21 pm
That's helpful, actually--it's a null pointer dereference.  All the zeroes under address are a dead giveaway.  Something isn't being initialized properly for whatever reason...

I'll investigate.
Title: Re: minisphere 1.0.8
Post by: N E O on April 11, 2015, 01:54:58 pm
Re MP3 (and certain other file types) support - would you like to give a try to using JS loaders like mp3.js for these?
Title: Re: minisphere 1.0.8
Post by: Fat Cerberus on April 11, 2015, 05:51:24 pm
That would of course be an option, although it would require me to expose some sort of raw stream object to script, which would deviate pretty sharply from the Sphere 1.x API.  Maybe for minisphere 2.0 (the planned Pegasus edition), but certainly not in this first version.
Title: Re: minisphere 1.0.8
Post by: Flying Jester on April 11, 2015, 07:57:23 pm
it would require me to expose some sort of raw stream object to script


An ArrayBuffer. The word you are looking for is ArrayBuffer.  :P  But It should be possible even with ye olde ByteArrays, though, shouldn't it?

We need to do a lot of work on Pegasus still. Right now it's still just an early draft. I agree with most of it, but there are lots of little points to iron out, and lots of stuff I don't think should be a part of the core API but rather an extension.
Title: Re: minisphere 1.0.8
Post by: Fat Cerberus on April 11, 2015, 08:36:38 pm
The ArrayBuffer is only part of it. That's just data after all--the engine itself still has to be able to know what to do with it. It's like saying you could implement networking entirely in script with just ByteArrays. :P

Also, I'm pretty sure Duktape doesn't support ArrayBuffers. (Not yet anyway.  The developer is working on it from what I understand...)

As for Pegasus and not being finalized: let me just say I despise design-by-committee.  I'd rather forge ahead now and work out the specifics later.  I don't really mind refactoring and we want to get people excited about Sphere 2.0, which isn't going to happen unless they see concrete evidence in the form of a working engine.

I mean look at ES6.  That's still a draft, but a lot of JS engines went ahead and implemented parts of it anyway.  Duktape has the Proxy object for example--which I took advantage of to implement ByteArrays in minisphere.  And seeing that in action in turn got me more excited about ES6.  I don't see why we can't do the same with Pegasus.
Title: Re: minisphere 1.0.8
Post by: Flying Jester on April 11, 2015, 09:53:42 pm

The ArrayBuffer is only part of it. That's just data after all--the engine itself still has to be able to know what to do with it. It's like saying you could implement networking entirely in script with just ByteArrays. :P



Fair enough. I still need to actually expose this from my audio plugin, too.



As for Pegasus and not being finalized: let me just say I despise design-by-committee.  I'd rather forge ahead now and work out the specifics later.  I don't really mind refactoring and we want to get people excited about Sphere 2.0, which isn't going to happen unless they see concrete evidence in the form of a working engine.


Sure. Most of my concerns could even be fixed just by changing what is required to be called an extension and what isn't :)
Title: Re: minisphere 1.0.9
Post by: Fat Cerberus on April 12, 2015, 02:13:20 am
1.0.9 is now available, now with FollowPerson(), function support for Set*Script, and a couple small fixes.
Title: Re: minisphere 1.0.9
Post by: DaVince on April 12, 2015, 08:40:23 am
I see you made FollowPerson() behave exactly like in Sphere 1.5, which is to say it takes the follower a bit of walking before suddenly warping in behind the leader. I always thought the FollowPerson() function was useless exactly *because* it behaves this way. Could you make it instead that the follower will try to catch up to the leader from whatever position they were already in when the FollowPerson() command was called?
Title: Re: minisphere 1.0.9
Post by: Fat Cerberus on April 12, 2015, 09:55:21 am
The problem there is that doing that requires built-in pathfinding, something I'm not too keen on building into the engine. I don't think that's really the function's intended purpose anyway--it's just a minimal implementation you can use to make your party members follow each other around.  Anything more advanced is left up to the game developer to do in script.

TL;DR incoming: See, I've learned something while working on this project, it's the thing that makes Sphere such an awesome engine: It provides a lot of built-in functionality, which allows you to get a game up and running very quickly, but it's also quite deliberately designed to get out of the way when you decide you want to step outside of its provided parameters.  You generally don't appreciate this as someone on the outside looking in developing a game, but seeing the finer points of how the engine does things under the hood, I've started to appreciate the little subtleties.

A great example: I had to fix edge script triggering not once, but twice because of this.  When I first implemented edge scripts, a cursory glance at the reference implementation (i.e. Sphere) yielded this code:
Code: (c++) [Select]
if (input_valid)
{
    if (!UpdateEdgeScripts())
        return false;
}

I naturally interpreted this to mean that edge script activation requires a valid input attachment (AttachInput).  I later found out this wasn't the case when the Trial and Error intro turned out to be broken in minisphere, and, after another quick look--this time at the UpdateEdgeScripts() function--changed it to require an active camera attachment instead.  This fixed T&E, but was still incorrect, which I didn't find out until Radnen sent me his Blockman game and I couldn't move between maps.  The game manages the camera itself in script, so there's no AttachCamera() calls to be found.  In other words, the edge script activation logic only cares about where the camera actually *is*, not who's holding it.  Realizing this was an epiphany: I suddenly understood that the engine was deliberately designed this way so that you could replace AttachCamera() with your own implementation if you wanted to, and that doing so wouldn't break other things that you might not necessarily want to replace.

Sphere doesn't look like much, but it's actually a pretty well-engineered little engine. :D

As for that input_valid variable?  Turns out what it actually means is whether the map engine is currently accepting input at all (i.e. are we inside the main MapEngine() loop), regardless of whether there's a current input attachment or not.  Poor choice of naming, which I of course rectified in minisphere (it's called is_main_loop there :P).
Title: Re: minisphere 1.0.9
Post by: DaVince on April 12, 2015, 12:11:09 pm
Quote
The problem there is that doing that requires built-in pathfinding, something I'm not too keen on building into the engine. I don't think that's really the function's intended purpose anyway--it's just a minimal implementation you can use to make your party members follow each other around.  Anything more advanced is left up to the game developer to do in script.


Well, I honestly was just thinking about solving it like "move toward player at twice or thrice the normal speed without regarding any collision" (since it disregards collision anyway) just so the transition of a character being where it is to arriving near the player makes more sense. I never expect FollowPerson() to be used when the follower isn't already at least near the leader, and there wouldn't be a point to the function at all when it's implemented with its current looks-like-it-is-bugged behavior. And I rather like the simple function and would use it if it wasn't for this one small nagging thing.

Also, that is an extremely cool story! Pretty neat way for the engine to figure out the right thing. :D
Title: Re: minisphere 1.0.9
Post by: Fat Cerberus on April 12, 2015, 01:40:23 pm
Well to be fair, it doesn't have to deal with collision since the followers just follow the exact path the leader does and outside of edge cases with vastly different base dimensions, they're guaranteed not collide with anything either.  This improves performance, which is important since the follow tracking has to move a lot of data around every frame itself.

But I do agree with the sentiment here.  I'll see what I can do.  Post an issue on GitHub about it so we can track it. :)
Title: Re: minisphere 1.0.9
Post by: DaVince on April 12, 2015, 02:56:06 pm
Done (https://github.com/fatcerberus/minisphere/issues/22). I also explained how I think this would work best while keeping it simple, in my opinion.
Title: Re: minisphere 1.0.9
Post by: Fat Cerberus on April 13, 2015, 02:00:38 am
Okay, functions-as-scripts is awesome and I have no idea why I didn't implement it sooner.  As part of my tests of SetDefaultPersonScript I wrote this bit of tomfoolery:

Code: (javascript) [Select]
SetDefaultPersonScript(SCRIPT_ON_ACTIVATE_TOUCH, function() {
food = GetCurrentPerson();
if (GetInputPerson() == 'maggie' && food != 'robert') {
new Scenario()
.playSound('Munch.wav')
.run();
DestroyPerson(food);
}
});


So much nicer than using a string for that; you don't get syntax highlighting for code in strings. :P  Also, yes: minisphere has SetDefaultPersonScript.  Sphere 1.5 apparently has it as well (I didn't think it did, but no unknown identifier errors when I call it), however it appears to be broken there as my test above didn't work in vanilla.  It works perfectly in minisphere. :D  So much eatiness...

Edit: Well, that explains that then.  Apparently Sphere's implementation of default person scripts is only partial: It only ever calls the default script for ON_CREATE, none of the other default person scripts are ever touched.  How did such an incomplete implementation like that ever make it into the official 1.5 release? ???
Title: Re: minisphere 1.0.9
Post by: Fat Cerberus on April 14, 2015, 02:13:41 am
Well, the next release is coming along.  This one is taking longer than usual since I want to get all the wrinkles ironed out first, as I'm planning for 1.0.10 to be the last 1.0 release before I start working on a few bigger changes for minisphere 1.1 (FollowPerson enhancements, config.exe...).

I finally fixed an audio deadlock bug that's been plaguing me since upgrading Allegro a few releases ago.  I assume nobody else ran into it as I received no bug reports of the sort, which I suppose is a good thing.  Here's what the changelog for 1.0.10 looks like thus far:

Code: [Select]
v1.0.10 - 
- Fixes IsKeyPressed() not recognizing modifier keys on right side of
  keyboard (#20)
- Improves SetPersonFrame compatibility (out-of-range frame is now
  wrapped instead of throwing an error)
- Fixes joystick assert error when running minisphere in OS X. (#19)
- Fixes wrong direction being rendered for 4-direction person sprites
  moving diagonally
- Fixes random deadlocks when playing sounds


So quite a few small but important fixes here.

Oh, also, @Radnen: Did you ever test out the SetPersonFrame fix I posted on GitHub?  I was waiting on you to test it before I merged it into master.
Title: Re: minisphere 1.0.9
Post by: Radnen on April 14, 2015, 10:00:02 am

Oh, also, @Radnen: Did you ever test out the SetPersonFrame fix I posted on GitHub?  I was waiting on you to test it before I merged it into master.


Yeah I saw it, I've literally had no time to test it. I'll test later today, sorry for the wait.
Title: Re: minisphere 1.0.9
Post by: Fat Cerberus on April 14, 2015, 03:28:02 pm
New plan for v1.1: a native x64 Windows build.  Now that I can compile Allegro myself on Windows, there's little stopping me from doing this. :)
Title: Re: minisphere 1.0.9
Post by: Flying Jester on April 14, 2015, 05:08:02 pm
It's always best to compile your dependencies yourself on Windows :)
Title: Re: minisphere 1.0.9
Post by: Radnen on April 14, 2015, 10:21:30 pm

Oh, also, @Radnen: Did you ever test out the SetPersonFrame fix I posted on GitHub?  I was waiting on you to test it before I merged it into master.


Yes, it fixed it. Also, minisphere is too much like sphere 1.5 in that it has the same animation bug that future boy nil fixed. You notice it in my blockman game when picking up bushes, rockets, etc. It has a glitchiness due to the last frame not being respected (when the last frame is met, make sure to go through it entire delay stack before switching over to frame 0).
Title: Re: minisphere 1.0.9
Post by: Fat Cerberus on April 14, 2015, 10:24:56 pm
What exactly is the bug?  Odd that I would have replicated it, as I made sure to always initialize the delay when switching frames.  There must be oddities with the animation setup that make it easy to screw up...
Title: Re: minisphere 1.0.9
Post by: Fat Cerberus on April 14, 2015, 11:06:45 pm
Hm, I think I found the "bug".

Blockman: 5 frames (excluding east/west which have 4)
Held object: 4 frames

SetPersonFrame() now correctly modulus-wraps the frame index, HOWEVER GetPersonFrame() returns the real current frame.  This is the source of the problem.  To illustrate:

Suppose Blockman is currently on the fourth frame of his animation:


Not really sure how FBN would have "fixed" this, as I wouldn't consider it a bug in the first place. ???
Title: Re: minisphere 1.0.9
Post by: Radnen on April 15, 2015, 12:03:33 am
It's with QueuePersonCommand and COMMAND_ANIMATE.

Say you have 5 frames at 8 delay frames each, you can loop through like this to animate a direction:
Code: (javascript) [Select]

for (var f = 0; f < 5; ++f) {
    for (var d = 0; d < 8; ++d) {
        QueuePersonCommand(COMMAND_ANIMATE);
    }
}


You'll notice in Sphere 1.5 you get a strange glitch in the animation where f == 4, we are at frame 0 for 8 frames. In Sphere 1.6 when f ==4, it correctly animates frame 5 for 8 frames.
Title: Re: minisphere 1.0.9
Post by: Fat Cerberus on April 15, 2015, 01:47:22 am
minisphere's COMMAND_ANIMATE implementation looks like this:

Code: (c) [Select]
switch (command) {
case COMMAND_ANIMATE:
    person->revert_frames = person->revert_delay;
    if (person->anim_frames > 0 && --person->anim_frames == 0) {
        ++person->frame;
        person->anim_frames = get_sprite_frame_delay(person->sprite, person->direction, person->frame);
    }
    break;
// ...
}


And for good measure, get_sprite_frame_delay:
Code: (c) [Select]
int
get_sprite_frame_delay(const spriteset_t* spriteset, const char* pose_name, int frame_index)
{
    const spriteset_pose_t* pose;
   
    if ((pose = find_sprite_pose(spriteset, pose_name)) == NULL)
        return 0;
    frame_index %= pose->num_frames;
    return pose->frames[frame_index].delay;
}


Looking at this code, I don't understand why this is happening, but I'm rather tired at the moment.  I'll have to investigate properly in the morning.
Title: Re: minisphere 1.0.9
Post by: Radnen on April 15, 2015, 01:49:55 am
Are you looking at Sphere 1.5's source or 1.6's? I'd compare the two to see what he did...
Title: Re: minisphere 1.0.9
Post by: Fat Cerberus on April 15, 2015, 01:51:09 am

Are you looking at Sphere 1.5's source or 1.6's? I'd compare the two to see what he did...


I have the one from sphere-group on GitHub, I assume that's 1.6?  That's what I've been using as reference this whole time.
Title: Re: minisphere 1.0.9
Post by: Radnen on April 15, 2015, 02:50:55 am
I thought the github branch had branches for 1.5/1.6, but to me it seems to be all of 1.6. Maybe that code is the corrected version? But your engine still has the Sphere 1.5 animation bug. Hmm...
Title: Re: minisphere 1.0.9
Post by: Flying Jester on April 15, 2015, 03:23:09 am

...but to me it seems to be all of 1.6.

I'm 99% sure that Sphere-Group/Sphere is 1.6, with some more work done after the official 1.6 beta binaries were made. Well, plus the work Raahkin and I did to make it work again on OS X and Linux, respectively. It's got the particle engine, the new sound effect object, and some other secret new stuff like the zlib API that shouldn't be in 1.5.
Title: Re: minisphere 1.0.9
Post by: Fat Cerberus on April 15, 2015, 10:36:55 am
...wow.  Just all I will say is "wow".  Still haven't found the cause of the animation bug, but I did find this:

Code: (c++) [Select]
for (unsigned int i = 0; i < m_Directions.size(); i++)
  if (strcmp_ci(direction.c_str(), m_Directions[i].name.c_str()) == 0)
  {
    d = i;

    // stop immediately.
    i = m_Directions.size();  // <-- wtf
  }


Traditionally most people early-terminate a for loop using a break statement...

Edit: Here are the results of my tests: COMMAND_ANIMATE seems to work fine.  Your Blockman sprite has a delay of 4 on each frame, so I did the following:
Code: (c++) [Select]
SetPersonSpriteset('robert', LoadSpriteset('blockman.rss'));
for (i = 0; i < 5; i++) for (j = 0; j < 4; ++j) {
Console.writeLine("Frame: " + GetPersonFrame('robert'));
Console.append("i = " + i);
Console.append("j = " + j);
FlipScreen();
QueuePersonCommand('robert', COMMAND_ANIMATE, false);
UpdateMapEngine();
}


It correctly cycled through all the frames (originally there was some log output here but it was a waste of space).  Sphere 1.5 was somewhat odd as it stayed on frame 0 twice as long the first time around (8 frames instead of 4), but otherwise cycled through all 5 images as normal.

Are you sure your issue wasn't with SetPersonFrame() and not COMMAND_ANIMATE?  Because that was the only difference I could find between 1.6 and minisphere, minisphere's SetPersonFrame wasn't resetting the animation frame counter.

Edit 2: Yeah, it was SetPersonFrame.  Sphere 1.5:
Code: (c++) [Select]
m_Persons[person].frame = frame;


Sphere 1.6:
Code: (c++) [Select]
m_Persons[person].frame = frame;
m_Persons[person].stepping_frame_revert_count = 0;
m_Persons[person].next_frame_switch = p.spriteset->GetSpriteset().GetFrameDelay(p.direction, p.frame);


1.6 resets the frame counter when the frame is set explicitly, 1.5 doesn't.  minisphere follows the 1.6 implementation now.
Title: Re: minisphere 1.0.9
Post by: Fat Cerberus on April 15, 2015, 02:28:51 pm
I just made my first x64 build of minisphere, and it's working great!  There's a good number of warnings due to conversions between int and size_t but other than that no issues that I've noticed so far.  Note that Allegro and every one of its dependencies generated similar warnings; whoever had the bright idea that both int and long should be 32-bit in 64-bit MSVC needs to be shot.

The best part?  The 64-bit build actually seems faster, especially noticeable with games that load a lot of resources upfront. :)
Title: Re: minisphere 1.0.10
Post by: Fat Cerberus on April 16, 2015, 01:45:06 pm
minisphere 1.0.10 is out.  I'm including the 64-bit build as an experimental surprise addition to 1.0.10 so that it can get some real-world testing before it's officially unveiled in minisphere 1.1. :)  Aside from that, this is basically a huge bugfix release to hopefully stabilize the 1.0 engine for real-world use.
Title: Re: minisphere 1.0.10
Post by: Fat Cerberus on April 16, 2015, 11:54:51 pm
So I had my first foray into 64-bit debugging in MSVC.  It was a simple 10-second fix for a null dereference, but boy was the exception dialog (shown below) disorienting.  I've been debugging 32-bit C/C++ apps for years, so I'm used to seeing this dialog, but all the extra digits in the memory addresses... wow.  It took me a good 20 seconds I think to recover from the shock, it was dizzying. :P
Title: Re: minisphere 1.0.10
Post by: Fat Cerberus on April 18, 2015, 01:52:02 am
Anyone have any suggestions for what they want to see in minisphere 1.1?  Almost my entire v1.1 checklist has been checked off except for implementing config.exe, so I figure I should solicit feature requests now. :)
Title: Re: minisphere 1.0.10
Post by: Radnen on April 18, 2015, 04:04:05 am
How do we check if we are running a game in minisphere? GetVersion() and GetVersionString() might have to take a parameter... not sure what though, here's the deal:

1. You can't overwrite the functionality of these functions because games check against them for compatibility, so they should return the API complacency. (v1.5 for the string and 1.5 for the number).
2. You can't pass 'true' to them to get minisphere's version since in SSFML, I could also return it's unique version too.
3. You pass a string, 'minisphere', to get minisphere's version. It returns nothing if you don't use minisphere. Trouble is, Sphere 1.5 will still return it's version number.

So... I guess the only way to detect sphere version is by feature detection. Namely we can use the features/apis function... whatever it was named.
Title: Re: minisphere 1.0.10
Post by: Fat Cerberus on April 18, 2015, 09:08:56 am
Check whether GetExtensions exists.  If it doesn't, that's stock Sphere.  If it does, call it and check if 'minisphere' is included in the extensions list.  That's why I added that function, and why I also added it to SSFML recently as well.

That said, do note that GetVersionString() in minisphere returns a string like this:
Code: [Select]
v1.5 (compatible; minisphere 1.0.10)


I've been doing that since minisphere's inception and haven't had a game break because of it yet.
Title: Re: minisphere 1.0.10
Post by: Fat Cerberus on April 18, 2015, 10:50:27 am
Just implemented a bunch of new APIs for follower management for 1.1.  FollowPerson was one of those dark corners of the Sphere map engine that really needed some love; heck, the only way to get a person's leader was through engine-provided metadata returned by GetPersonData, and the rest of the info was just unavailable.  minisphere 1.1 will add the following APIs:



I have to thank DaVince for pointing out deficiencies in the Sphere implementation, or it never would have received these enhancements.  You can now write a handler, for instance, that condenses follower chains when a person in the middle of the chain is destroyed:
Code: (javascript) [Select]
SetDefaultPersonScript(SCRIPT_ON_DESTROY, function() {
var person = GetCurrentPerson();
var distance = GetPersonLeader(person) != "" ? GetPersonFollowDistance(person) : 0;
Link(GetPersonFollowers(person)).each(function(name) {
FollowPerson(name, GetPersonLeader(person), distance);
});
});
Title: Re: minisphere 1.0.10
Post by: Radnen on April 18, 2015, 12:33:03 pm

That said, do note that GetVersionString() in minisphere returns a string like this:
Code: [Select]
v1.5 (compatible; minisphere 1.0.10)


I've been doing that since minisphere's inception and haven't had a game break because of it yet.


I guess my fear there is someone is checking against the string. So far there hasn't been an issue tho. Yes, GetExtensions was what I was thinking about, I guess it's the only reliable solution.
Title: Re: minisphere 1.0.10
Post by: Fat Cerberus on April 18, 2015, 12:38:45 pm
Yeah, I would have changed the string back by now if there were a compatibility issue.  From what I've observed, games that want to detect version level use GetVersion() instead, which returns 1.5 as always.  Which makes sense, since then you can use comparison operators on it:

Code: [Select]
if (GetVersion() < 1.5) {
  Abort("Unsupported engine version!");
}
Title: Re: minisphere 1.0.10
Post by: DaVince on April 18, 2015, 01:50:30 pm


That said, do note that GetVersionString() in minisphere returns a string like this:
Code: [Select]
v1.5 (compatible; minisphere 1.0.10)


I've been doing that since minisphere's inception and haven't had a game break because of it yet.


I guess my fear there is someone is checking against the string. So far there hasn't been an issue tho. Yes, GetExtensions was what I was thinking about, I guess it's the only reliable solution.

Even if there is, it's their problem to solve. They wrote the game for Sphere 1.5, and it's pretty obvious that not every single stock 1.5 game works out of the box with minisphere (for example, any using the const keyword, or any using MP3 files).
Title: Re: minisphere 1.0.10
Post by: Radnen on April 18, 2015, 02:30:43 pm

Even if there is, it's their problem to solve. They wrote the game for Sphere 1.5, and it's pretty obvious that not every single stock 1.5 game works out of the box with minisphere (for example, any using the const keyword, or any using MP3 files).


Yeah, I guess you're right. New games shouldn't have that problem. I already modified Blockman to run in minisphere, I guess old games are just stuck with old engines.
Title: Re: minisphere 1.0.10
Post by: Flying Jester on April 18, 2015, 05:18:26 pm
I suggest adding SetTileName(). I added that to my map engine just because I thought it was odd it was missing, and it was trivial to add.
Title: Re: minisphere 1.0.10
Post by: Fat Cerberus on April 18, 2015, 05:19:58 pm
Yeah, that would be trivial to add.  Is there a valid use case for it though?  I try not to add APIs just for the sake of it, trying to minimize perceived bloat.
Title: Re: minisphere 1.0.10
Post by: Flying Jester on April 18, 2015, 05:39:43 pm
I'm not sure it has a proper need, but every other method that has a Get also has a Set. I added it just for consistency.
Title: Re: minisphere 1.0.10
Post by: Radnen on April 18, 2015, 05:46:21 pm

Yeah, that would be trivial to add.  Is there a valid use case for it though?  I try not to add APIs just for the sake of it, trying to minimize perceived bloat.


GetTileName() is impossible otherwise, keep that in mind too.

Actually, if you want to reduce bloat, then it'd be nice to make a method where you get a POJO (plain old javascript object) of the map and it's tileset. You can then modify said POJO, re-update the map, and see the changes.

We can create Sphere 1.5 complacent wrappers around that POJO. This will ad 2 new calls to the map engine and literally remove anywhere between 20 and 50 API calls from Sphere.

Going one step further, it'd be neat to store the map POJO as JSON anyways. It has the added benefit of removing the need for analogue/persist for good (because you can modify the map state and save it into a separate save file with little-to-no effort).

That way, we can create either an in-engine way of handling save games, or someone can make an efficient, and easy to use save system for your games (person entities, maps, etc. are stored as POJO). I say in-engine since many new programmers struggle with saving a game and if we had a small API that's also very powerful, it could aid in bringing people to use minisphere.
Title: Re: minisphere 1.0.10
Post by: Fat Cerberus on April 18, 2015, 05:50:10 pm
While it's a neat idea, I think it's outside the scope of minisphere 1.x.  I might consider it for 2.0, however. :)

Also: you don't need SetTileName for GetTileName to work, can't you set tile names in the editor?  I'll implement it anyway though, it'll literally take about 30 seconds.
Title: Re: minisphere 1.0.10
Post by: Flying Jester on April 18, 2015, 05:53:45 pm

Actually, if you want to reduce bloat, then it'd be nice to make a method where you get a POJO (plain old javascript object) of the map and it's tileset. You can then modify said POJO, re-update the map, and see the changes.

We can create Sphere 1.5 complacent wrappers around that POJO. This will ad 2 new calls to the map engine and literally remove anywhere between 20 and 50 API calls from Sphere.

Going one step further, it'd be neat to store the map POJO as JSON anyways. It has the added benefit of removing the need for analogue/persist for good (because you can modify the map state and save it into a separate save file with little-to-no effort).


This is exactly what my map engine does anyway since it is in script, and I can tell you it really is the way to go. More than half of the code by lines of my map engine is 1.5 functions, and even then I condensed them.
Doing the same for tilesets, spritesets, and persons in particular is a really good idea, too.

Of course, I think the map engine is best done in script anyway...but even so, I suspect just having the 1.5 functions in script at least would go a long way to simplifying the implementation.

This would also encourage the idea of setting up a map object and then running it, instead of weirdly spooling objects, scripts, and persons. I believe this is the ugliest part of the map engine.
Title: Re: minisphere 1.0.10
Post by: Fat Cerberus on April 18, 2015, 07:42:36 pm
I used to hate the Sphere map engine with a passion, but it's odd--reimplementing it from scratch, I've developed a certain fondness for it.  It's elegant, in a "diamond in the rough" kind of way.

I think my only real issue with the way the map engine is set up, is the unnecessary dichotomy between "map engine running" and "outside the map engine" caused by the fact that MapEngine() runs its own update loop.  Every other issue (API, etc.) is merely cosmetic in my eyes.

Really though, I've begun to see the legacy API as Sphere's biggest strength (we've made it this far with it, after all, haven't we?).  Some of the things it does seem... off to a seasoned JS coder (referencing persons by name, loose coupling between persons and map, etc.) but are nonetheless intuitive to someone who just wants to get a project off the ground quickly.  Certainly more so than the morass of objects necessitated by the Pegasus spec.

That's not to say I view Pegasus as worthless, of course--quite the opposite, it's very much a step in the right direction.  I just don't think we need to write off the legacy API as a second-class citizen--it's quite valuable in its own right.
Title: Re: minisphere 1.0.10
Post by: Flying Jester on April 18, 2015, 08:31:58 pm
Well, I originally wanted to remake the map engine so that the demo would look like this:

Code: (javascript) [Select]
function game(){
    var aegis = new Person("Aegis", false);
    var map = new Map("map.rmp");
    map.addPerson(aegis);
    MapEngine(map, 60);
}


It wouldn't be too hard I think to allow this, too, though:

Code: (javascript) [Select]
function game(){
    /* Rather than `MapEngine(60);' */
    while(RunEngine(60));
}


I still think that style would be better. Being able to call up the list of persons, or just get a person by name, would still be a reasonable thing to do using that style.
It wouldn't even be too difficult to allow passing in a name string optionally instead of a person object.

I agree that the Pegasus spec still needs work. A lot of it overly complex. I also agree that the old API is basically a good design. That's why I've only removed the old old drawing API in TurboSphere. The only major changes I'm really sold on so far is namespacing the API and adding basic OOP to it. Not anywhere near the OOP madness that Pegasus has, just small things, things that should really be properties instead of getters like RawFile.getSize(), and using real constructors.
Title: Re: minisphere 1.0.10
Post by: Radnen on April 18, 2015, 09:17:51 pm
The way I see it, the Sphere 1.5 API is perfect in every step of it's existence, but if you think about it, it doesn't need to be implemented in the engine. Rather, only a few API calls ever need to be exposed and that's the direct interaction with each file type and input/output method. So really you only need like a couple dozen API calls.

1. Maps/People/Spritests/etc. are full JS objects that you can create the Get/Set API from, script side.
2. All objects become normal JS constructors, which you can make the Load*() API calls from.
3. All graphics are ran through a vertex/shape API that you can make the primitives and images API from.
4. All input is handled by global singletons for that game instance, where you can make the input API from (keyboard, mouse, joystick).
5. All files use 1 IO API which you can make all LogFile/RawFile/File API's from.

If you really think about it you only need to expose like 12 to 20 actual, hard coded, necessary API calls to sphere in order to run 90% of the rest of the API. Graphics, persons + maps, IO, input just have to use the few objects exposed to code and that's it.

It'll truly 'unlock' Sphere. It's great because you can then do cool things like:
Code: (javascript) [Select]

// we may have an issue here: how do you attribute people to maps not yet loaded?
function CreatePerson(name, ss, destroy) {
    var p = new Person(name, ss, destroy);
    _current_map.addPerson(p);
    return p;
}

function SetPersonX(name, x) {
    _current_map.people[name].x = x;
}

var p = CreatePerson("fred", "fred.rss", false);
p.items = [];
p.coins = [];

// what a naive implementation of Sphere File Object looks like:
function LoadFile(filename) {
    var p = new File(filename); // basically a raw file, and the only file type ever.
    p.write = function(key, value) { this.writeLine(key + '=' + value); }
    p.read = function(key, default) { // naive method
        var l = this.readLine(); // returns null if EOF
        while (l) {
            var data = l.split('=');
            if (data[0] == key) {
                if (typeof default == 'number') return Number(data[1]);
                if (typeof default == 'string') return data[1];
                if (typeof default == 'boolean') return Boolean(data[1]);
            }
            l = this.readLine();
        }
        return default;
    }
}

// easiest method ever to saving a person
var file = LoadFile("save.txt");
file.write("fred", JSON.stringify(p));
file.close();


TurboSphere is headed this way with it's Sphere 1.* shim. This is really a good idea. Having very few Sphere API calls also increases the fact you get more 'freedom' from the engine. I used to think you could never make a game without Sphere MapEngine... oh how wrong was I. It was just that since it was there, you had access to it and since there was this huge API around it, it made it hard to give the illusion of freedom, but it's there, you had a lot of freedom, but the API never always showed it. Especially when you hit brick walls like enumerating the fields of an image, adding a property to an image, or heck, even getting the dang filename of an image.
Title: Re: minisphere 1.0.10
Post by: Fat Cerberus on April 18, 2015, 09:31:31 pm
Haha, I love that comment on the CreatePerson() function.  From a purely theoretical standpoint, the Persons API doesn't seem to make much sense, does it?  That is, until you actually go to implement it.  Maps and persons in Sphere are actually very loosely coupled, which is big part of what makes the system so flexible.  The tendency I think is to always want to view persons as being associated with a given map, when in fact they NEVER are, the map engine just happens to conveniently destroy some of them for you when switching maps.
Title: Re: minisphere 1.0.10
Post by: Radnen on April 18, 2015, 09:55:11 pm

Haha, I love that comment on the CreatePerson() function.  From a purely theoretical standpoint, the Persons API doesn't seem to make much sense, does it?  That is, until you actually go to implement it.  Maps and persons in Sphere are actually very loosely coupled, which is big part of what makes the system so flexible.  The tendency I think is to always want to view persons as being associated with a given map, when in fact they NEVER are, the map engine just happens to conveniently destroy some of them for you when switching maps.


That just goes to show just how much was hidden behind the scenes, the person manager, the command queue, the way edge scripts are detected(!), and the camera, and the player input. Bringing these hidden areas to the forefront can increase productivity, these can be safely exposed in an extended API. Older games just wouldn't know to call these extended functions, but newer games can, and take full advantage of them. I'm still impressed to this day FBNil's ability to know how sphere operates behind the scenes to take advantage of person scripts, delay scripts, etc. in his lithonite engine.
Title: Re: minisphere 1.0.10
Post by: Fat Cerberus on April 18, 2015, 10:04:59 pm
That's what made getting the Lithonite demo fully working in minisphere such an awesome moment.  minisphere's map engine implementation was as much reverse engineering and extrapolation as it was directly studying Sphere's code.  Which ultimately was a good thing--I can only take so much of the latter activity before it starts to wear on my sanity.  Read the post at the top of page 22 for just one of the many reasons why that is. :P 

But yeah, I was able to extrapolate a lot of the map engine's inner workings from studying Lithonite's tricks.  It was quite fun, actually. :D
Title: Re: minisphere 1.0.10
Post by: Fat Cerberus on April 19, 2015, 02:22:09 am
Quick warning to anyone using minisphere for production use: Don't try to to pass a function to SetPersonScript, it will crash.  I accidentally passed a Duktape context to the internal set_person_script function instead of a person object, so it ends up overwriting Duktape state and everything goes to hell.  SetPersonScript() with a code string works fine, though, so I won't put out a hotfix for it.  It'll be fixed in 1.1.
Title: Re: minisphere 1.0.10
Post by: casiotone on April 19, 2015, 05:00:25 am
How about a DoesFileExist function? Not sure why it wasn't added with the other file/directory listing functions.
Title: Re: minisphere 1.0.10
Post by: DaVince on April 19, 2015, 08:39:08 am
^ This. I have severely missed this several times while making games.
Title: Re: minisphere 1.0.10
Post by: DaVince on April 19, 2015, 08:42:32 am
Really liking this recent discussion. :)


Yeah, that would be trivial to add.  Is there a valid use case for it though?  I try not to add APIs just for the sake of it, trying to minimize perceived bloat.

I can think of good use cases for SetTileName(). First off, of course there's the fact that once you save a map it'll have this changed tile name data in it. But what about tiles in your game that you want to change the behavior on? For example, tiles with the name "hazard" all behaving like hazards until you tell some tiles they no longer have this name and thus no longer are hazards.

I'm sure there are more not-quite-straight-forward use cases. It's nice that this has been implemented now, anyway.
Title: Re: minisphere 1.0.10
Post by: Fat Cerberus on April 19, 2015, 09:10:04 am

How about a DoesFileExist function? Not sure why it wasn't added with the other file/directory listing functions.


Already done.  In fact it should be in 1.0.10 already.  I ended up adding it after looking through the commit log on the old sourceforge CVS repo (when I was trying to get the 1.5 source to fix that animation bug) and seeing it.  I think it might not have actually been exposed to script though?  Anyway, yeah, minisphere has DoesFileExist() already. :)
Title: Re: minisphere 1.0.10
Post by: Fat Cerberus on April 19, 2015, 01:30:10 pm
Just added "proper object constructors" to the to-do list for 1.1.  :D
Title: Re: minisphere 1.0.10
Post by: casiotone on April 19, 2015, 02:51:41 pm
Awesome stuff. A way to enable filtering for images would be good.
Title: Re: minisphere 1.0.10
Post by: Fat Cerberus on April 19, 2015, 03:05:13 pm

Awesome stuff. A way to enable filtering for images would be good.

Not sure what you mean by this, you'll need to clarify.
Title: Re: minisphere 1.0.10
Post by: casiotone on April 19, 2015, 04:34:06 pm
https://www.allegro.cc/manual/5/al_set_new_bitmap_flags

ALLEGRO_MIN_LINEAR and ALLEGRO_MAG_LINEAR. It improves image scaling a lot.
Title: Re: minisphere 1.0.10
Post by: Fat Cerberus on April 19, 2015, 04:39:54 pm
Ah, bilinear filtering for scaling.  That'd be easy enough to add an API function or two for.
Title: Re: minisphere 1.0.10
Post by: Fat Cerberus on April 20, 2015, 01:33:14 am
I just started working on adding constructors for Sphere objects.  I posted an issue on GitHub to track the progress of the implementation.
Title: Re: minisphere 1.0.10
Post by: Flying Jester on April 20, 2015, 05:22:09 pm
I thought of two other things that I find useful additions to the API. These are additions I added for use by my map engine, so I have an actual use for them as opposed to SetTileName.

RawFile.read(), with no argument, reads the entire file.
RawFile.readString() works the same as read, but returns a string. This can save on memory usage, when otherwise an ArrayBuffer or ByteArray would have to be garbage collected. Until then, you have at least two entire copies of the read in memory.
Title: Re: minisphere 1.0.10
Post by: Fat Cerberus on April 20, 2015, 05:25:27 pm

I thought of two other things that I find useful additions to the API. These are additions I added for use by my map engine, so I have an actual use for them as opposed to SetTileName.

RawFile.read(), with no argument, reads the entire file.
RawFile.readString() works the same as read, but returns a string. This can save on memory usage, when otherwise an ArrayBuffer or ByteArray would have to be garbage collected. Until then, you have at least two entire copies of the read in memory.


RawFile.readString I probably would have done myself at some point.  I already did that for sockets, in fact.  :D  I like the idea of .read() with no args slurping the whole file, it saves the step of getting the length first.  Thanks!

As for garbage collection, do note that that's not as big an issue in minisphere as Duktape's primary GC method is refcounting.  It only resorts to mark and sweep in case of circular refs.
Title: Re: minisphere 1.0.10
Post by: Fat Cerberus on April 20, 2015, 08:11:10 pm
Let me just say how awesome it is to be able to do this:

Code: [Select]
var red = new Color(255, 0, 0);
Abort(red instanceof Color);  // true!


You have no idea how much and how often I wished to be able to do that in Sphere.  Instead all we got was the clunky toString() methods for type checking.

;D

instanceof works with the old-style methods too!
Title: Re: minisphere 1.0.10
Post by: DaVince on April 21, 2015, 06:02:40 pm
So I noticed a missing effect in the Sir Boingers intro screen: the backgrounds are supposed to fade in and out to and from black, but they're not. They're scrolling and a new background is shown without any transition whatsoever.

The fade effect is performed by overlaying a fullscreen black rectangle that changes its opacity - code at line 465-481 in start.js. For some reason this doesn't seem to render in minisphere (and I made sure to compile the latest version - works full speed even on an old laptop, good work!)
Title: Re: minisphere 1.0.10
Post by: Fat Cerberus on April 21, 2015, 06:06:25 pm
Thanks for the report, I'll look into it. :)
Title: Re: minisphere 1.0.10
Post by: Fat Cerberus on April 22, 2015, 02:33:08 am
Well, this has been a massive undertaking, but definitely worth it.  In minisphere 1.1, all Sphere objects will have proper constructors and, where applicable, actual properties in addition to the old-style get/set methods.  I even took some cues from TurboSphere when implementing a few of the constructors. :D  Amazingly, the engine executable hasn't grown much, if at all.  The 32-bit engine is still 1.5MB and 64-bit 1.75MB.  Pretty awesome.

I'll be honest though, if I had tried to implement things this way from the start, I can guarantee it would have been a failure.  minisphere 1.0 had to be what it was, a drop-in Sphere 1.5 replacement, legacy warts and all.  It gave me something to work towards, the entire existing Sphere catalog.  If I had gone for the modern approach right out of the gate, it wouldn't have been compatible with anything and would have gotten very boring very fast since the only thing I would have had is, at best, a bunch of contrived tests.  But now that I have that compatibility, I can freely build on it to create something great. ;D
Title: Re: minisphere 1.0.10
Post by: Fat Cerberus on April 22, 2015, 02:55:22 am

So I noticed a missing effect in the Sir Boingers intro screen: the backgrounds are supposed to fade in and out to and from black, but they're not. They're scrolling and a new background is shown without any transition whatsoever.

The fade effect is performed by overlaying a fullscreen black rectangle that changes its opacity - code at line 465-481 in start.js. For some reason this doesn't seem to render in minisphere (and I made sure to compile the latest version - works full speed even on an old laptop, good work!)


Apparently the copy of Sir Boingers I have doesn't have this effect?  I have the one from the Downloads repo, and the lines in question look like this:
Code: (javascript) [Select]
      while (AreKeysLeft()) key = GetKey();

      if (key != undefined)
      {
        if (key == KEY_BACKSPACE)
          playername = playername.slice(0, playername.length-1);
        else if (key == KEY_ENTER && !lastpressed && playername != "")
          break;
        else if (playername.length <= 20)
          playername += GetKeyString(key, shift);
      }

      key = "";

      if (IsAnyKeyPressed()) lastpressed = true;
      else lastpressed = false;
      if (IsKeyPressed(KEY_SHIFT)) shift = true;


Input code.  I did a search for all Rectangle() calls in start.js too, they all used a constant color.  No ApplyColorMask() calls either.  This effect must be a recent addition.

That said, drawing a full-screen translucent rectangle should definitely work, short of a regression in the primitives code, which I don't think I've touched in quite a while...
Title: Re: minisphere 1.0.10
Post by: DaVince on April 22, 2015, 06:02:37 am
Oh! I never noticed that Boingers 2.0/2.1 isn't on the repo. I'll post it soon. The game's expanded a fair bit!
Edit: now on the repo and http://boingers.tengudev.com/

Edit: also, lines 459-473. 472 especially. I referenced line numbers from my WIP next version, silly me.
Title: Re: minisphere 1.0.10
Post by: Fat Cerberus on April 22, 2015, 09:47:04 am
Yeah, I see the Rectangle() call.  I added some prints before the primitive call and used a debug build of minisphere (which shows the console window), the alpha values are correct, but the Rectangle apparently isn't being rendered.  Definitely something weird going on here.
Title: Re: minisphere 1.0.10
Post by: Fat Cerberus on April 22, 2015, 11:59:05 am
So I took a closer look at the print() output and figured out the issue: The alpha values on your mask rect are negative.  minisphere clamps colors to [0-255].  Sphere modulo's them, which I chose not to emulate because it screws up color tweening.  You can fix the effect by changing the alpha math to ensure the alpha is always in the range of [0-255].  It's not a bug, it's a feature! ;)

Also, tip: If you put a 32x32 icon.png in the project root, minisphere will use it for the game's taskbar icon. :)
Title: Re: minisphere 1.0.10
Post by: Fat Cerberus on April 22, 2015, 02:03:51 pm
So, um... anyone want to take 1.1 out for a test drive? ;)  I just posted minisphere 1.1b1.
Title: Re: minisphere 1.0.10
Post by: DaVince on April 22, 2015, 02:25:18 pm

So I took a closer look at the print() output and figured out the issue: The alpha values on your mask rect are negative.  minisphere clamps colors to [0-255].  Sphere modulo's them, which I chose not to emulate because it screws up color tweening.  You can fix the effect by changing the alpha math to ensure the alpha is always in the range of [0-255].  It's not a bug, it's a feature! ;)

Yeah, I've sometimes made use of the fact that you can go beyond 255 and the color just loops.
I'm not sure I even realized the number was going into the negative back when I coded that. Maybe I did. Anyway, I'll have to figure it out. Maths is not my strong point. :(

Quote
Also, tip: If you put a 32x32 icon.png in the project root, minisphere will use it for the game's taskbar icon. :)

Will do! For the next version, though, this one was just a bug fix release!
Title: Re: minisphere 1.1b1 (stable: 1.0.10)
Post by: Fat Cerberus on April 22, 2015, 02:48:46 pm
Admittedly it was a clever trick--send the alpha into negative to "reverse polarity" so to speak-- (-1) = 255, (-255) = 0.  Unfortunately it won't work in minisphere. :P. I'll only go so far in the name of compatibility!
Title: Re: minisphere 1.1b1 (stable: 1.0.10)
Post by: Fat Cerberus on April 22, 2015, 10:36:33 pm
@DaVince: Let me just say I love the snarky remarks Boingers 2.0 gives when you die.  The new levels were too hard for me, so I started playing the "old" level set - my favorite had to be 6 levels and about 20 deaths in, it told me "Well done" when I died and I cracked up. ;D  The game seems very stable on 1.1b1, no crashes, errors or anything.

Oh, by the way:
Code: (javascript) [Select]
if (imgtime < 48) mask.alpha = 255-(imgtime/48)*255;
if (imgtime > imgmaxtime-48) mask.alpha = 255-((imgmaxtime-imgtime)/48)*255;


As for minisphere, I've been debating a slight name change to "miniSphere".  You know, seeing as people seem to want to call it that anyway. ;)  Everyone's thoughts?
Title: Re: minisphere 1.1b1 (stable: 1.0.10)
Post by: Radnen on April 22, 2015, 10:54:04 pm
minisphere all lowcase shows it to be a light weight C built app. miniSphere sounds like a method to a JS function, and MiniSphere sounds like a legacy C++ or .NET app. :P
Title: Re: minisphere 1.1b1 (stable: 1.0.10)
Post by: Flying Jester on April 23, 2015, 12:15:46 am
miniSphere looks weird to me...

mini.js would be catchy, even if it didn't reflect the Sphere lineage.
Title: Re: minisphere 1.1b1 (stable: 1.0.10)
Post by: Fat Cerberus on April 23, 2015, 12:42:03 am

miniSphere looks weird to me...

mini.js would be catchy, even if it didn't reflect the Sphere lineage.


Nah, too much JS-related stuff uses the .js nomenclature (Node.js immediately comes to mind), it's a dumb fad as far as I'm considered.  It would be like me calling minisphere 'minisphere.c'.  Admittedly that works better if it were actually all in one .c file (I used to do that way back in the day too :P), but...

But yeah, miniSphere does seem odd.  I'll keep the name as minisphere.
Title: Re: minisphere 1.1b1 (stable: 1.0.10)
Post by: Flying Jester on April 23, 2015, 02:03:20 am
I got the idea from node.js...

But I agree. Node.js has a really dumb name, I think. I expect it to be written in JS with a name like that! It's like how when something is cname, I expect it to be written in C, like cpython.
Title: Re: minisphere 1.1b1 (stable: 1.0.10)
Post by: Fat Cerberus on April 23, 2015, 02:20:09 am
So I've been thinking about the config tool I'm planning to add in 1.1, and it strikes me that almost no existing Sphere game actually uses the key mappings set using it (i.e. the GetPlayerKey() function).  Thinking about why that would be, it strikes me that the likely reason is this: Sphere's config tool is, quite frankly, an atrocity.  You have the engine for normal gameplay, but if you want to change keys, plugins, etc., you have to close the engine, run config, and then run the game again.

This whole state of affairs is a shame, as one of the many great things about Sphere is how much tedium it offers to handle for you without getting in your way if you want to do things differently.  So me, not one to let a useful engine convenience go to waste, I came up with the following idea: An in-engine configuration screen.  I could even make it a little pulldown strip at the top of the screen so you can change settings without interrupting gameplay.  I could even allow per-game config this way!
Title: Re: minisphere 1.1b1 (stable: 1.0.10)
Post by: Radnen on April 23, 2015, 02:38:39 am

This whole state of affairs is a shame, as one of the many great things about Sphere is how much tedium it offers to handle for you without getting in your way if you want to do things differently.  So me, not one to let a useful engine convenience go to waste, I came up with the following idea: An in-engine configuration screen.  I could even make it a little pulldown strip at the top of the screen so you can change settings without interrupting gameplay.  I could even allow per-game config this way!


I agree. +1
Title: Re: minisphere 1.1b1 (stable: 1.0.10)
Post by: DaVince on April 23, 2015, 06:50:16 am
Always so much new stuff every day I check the thread. :)


@DaVince: Let me just say I love the snarky remarks Boingers 2.0 gives when you die.  The new levels were too hard for me, so I started playing the "old" level set - my favorite had to be 6 levels and about 20 deaths in, it told me "Well done" when I died and I cracked up. ;D  The game seems very stable on 1.1b1, no crashes, errors or anything.

Oh, by the way:
Code: (javascript) [Select]
if (imgtime < 48) mask.alpha = 255-(imgtime/48)*255;
if (imgtime > imgmaxtime-48) mask.alpha = 255-((imgmaxtime-imgtime)/48)*255;


Thanks! And ha, yeah, that's exactly what I did. Thanks though! :)

Quote
So I've been thinking about the config tool I'm planning to add in 1.1, and it strikes me that almost no existing Sphere game actually uses the key mappings set using it (i.e. the GetPlayerKey() function).  Thinking about why that would be, it strikes me that the likely reason is this: Sphere's config tool is, quite frankly, an atrocity.  You have the engine for normal gameplay, but if you want to change keys, plugins, etc., you have to close the engine, run config, and then run the game again.

This whole state of affairs is a shame, as one of the many great things about Sphere is how much tedium it offers to handle for you without getting in your way if you want to do things differently.  So me, not one to let a useful engine convenience go to waste, I came up with the following idea: An in-engine configuration screen.  I could even make it a little pulldown strip at the top of the screen so you can change settings without interrupting gameplay.  I could even allow per-game config this way!

Yes please! And it'd be even nicer if you exposed some of the config settings through the API too, like the key bindings.

Also, input is kind of a complex beast. I was writing a long post asking for more full keymapping support that is analogous to modern gamepads, but thinking about it, this would NEED to come with a way to save the configuration in the game folder itself (and include it as a config in game.sgm or a config.ini in the same folder or something). Because some games have mappings where WASD moves you and the arrow keys do other stuff (or you need to use the mouse) while others would use the arrow keys.

It's probably a good idea to have the global config file which saves things like window scale and filter, and the game-specific section saves stuff relevant for only this game (like the input). I also think you can drop/not implement some stuff like disabling sound, because honestly, OSes just support disabling sound on individual apps now.

Oh! And sometimes you could expect different behavior from the keyboard and mouse compared to gamepads. You will want to stick with the raw keyboard input then. Sir Boingers is set up so that it both accepts the joystick API and keyboard input, but the keyboard input is all done with Sphere's keyboard-related functions because the game simply behaves differently when played with a keyboard. For example, on a keyboard, "up" makes you float, while with a controller it uses other buttons because that's easier.
However, I consider this a "don't worry about it" case as I've set up the game to check both the keyboard and gamepad anyway, and so should anyone else with controls that aren't the same on different controllers. They're more specialized cases.
Title: Re: minisphere 1.1b1 (stable: 1.0.10)
Post by: Fat Cerberus on April 23, 2015, 10:14:54 am

Yes please! And it'd be even nicer if you exposed some of the config settings through the API too, like the key bindings.

They kind of are though?  That's what GetPlayerKey() is, but nobody uses it because the current config.exe is clunky (and huge--take a look at a Sphere 1.5 distro some time--config.exe is literally the biggest thing there, even bigger than the editor! :o).  Unless you mean an API to set the mappings, in which case... eh.  I think at the point your game wants to do that, it means you're already exposing your own configuration UI anyway, so might as well go all the way and manage the settings yourself.  Let's not invite scope creep here. :)

Quote

Also, input is kind of a complex beast. I was writing a long post asking for more full keymapping support that is analogous to modern gamepads, but thinking about it, this would NEED to come with a way to save the configuration in the game folder itself (and include it as a config in game.sgm or a config.ini in the same folder or something). Because some games have mappings where WASD moves you and the arrow keys do other stuff (or you need to use the mouse) while others would use the arrow keys.


The current config GetPlayerKey() mappings are already based on a gamepad (an SNES pad, to be specific: a single D-pad with four face buttons and a Start button--Select is MIA for some reason), so it would be easy enough to expand the mappings.  The thing is though, again, Sphere's design is set up primarily for making retro games, so the conveniences provided by the engine reflect that.  Anything more advanced, in my opinion, should be done in script.  This kind of thing is why the File API exists (referring to the .ini-style File object, not RawFiles), after all.

Quote

It's probably a good idea to have the global config file which saves things like window scale and filter, and the game-specific section saves stuff relevant for only this game (like the input). I also think you can drop/not implement some stuff like disabling sound, because honestly, OSes just support disabling sound on individual apps now.

Oh! And sometimes you could expect different behavior from the keyboard and mouse compared to gamepads. You will want to stick with the raw keyboard input then. Sir Boingers is set up so that it both accepts the joystick API and keyboard input, but the keyboard input is all done with Sphere's keyboard-related functions because the game simply behaves differently when played with a keyboard. For example, on a keyboard, "up" makes you float, while with a controller it uses other buttons because that's easier.
However, I consider this a "don't worry about it" case as I've set up the game to check both the keyboard and gamepad anyway, and so should anyone else with controls that aren't the same on different controllers. They're more specialized cases.


I actually don't like the window scaling being global.  It's one of my biggest peeves with Sphere 1.5--I generally keep 2x scale on because the majority of Sphere games are 320x240, but if I happen to come across one that uses 640x480 (or higher--T&E is 800x600!), the window is bigger than the screen and then I have to reconfigure it, which shrinks all my 320x240 games in the process.

Oh, and speaking of Sir Boingers, I'm actually not a big fan on the new control setup in v2.  The 2010 version's setup was better, I think (Ctrl to bounce).  A big issue with the new controls I've had is that, when bouncing and trying to move left and float at the time, the Up input gets ignored because of simultaneous-key limitations on cheaper keyboards (the keyboard on my Core i3 2-in-1 laptop, for example).  This makes a few areas almost impossible and cost me a good number of avoidable deaths.  I think this is why old console emulators used to use Ctrl and Alt for the face buttons, actually, since modifier keys don't contribute to the 3-key limit.
Title: Re: minisphere 1.1b1 (stable: 1.0.10)
Post by: mezzoEmrys on April 23, 2015, 10:22:42 am
From what I'd expect, all to almost all of the configuration options should be in a built in menu bar of some kind, like when using Visual Boy Advance, so you can change all those options on the fly. It makes a lot more sense to me, but I'm not sure how feasible it would be to change which graphics and audio plugins you're using on the fly.
Title: Re: minisphere 1.1b1 (stable: 1.0.10)
Post by: Fat Cerberus on April 23, 2015, 10:24:44 am

From what I'd expect, all to almost all of the configuration options should be in a built in menu bar of some kind, like when using Visual Boy Advance, so you can change all those options on the fly. It makes a lot more sense to me, but I'm not sure how feasible it would be to change which graphics and audio plugins you're using on the fly.


Yeah, that's what I was planning here, ability to change the configuration without stopping the game you're playing.  Plugins aren't an issue as minisphere's backend isn't customizable--it's always Allegro. :)
Title: Re: minisphere 1.1b1 (stable: 1.0.10)
Post by: Fat Cerberus on April 23, 2015, 12:53:10 pm
Well, I decided on a whim that 1.1 doesn't represent a big enough evolution and am now going to implement the Sapphire API from TurboSphere.  After looking at the method signatures and such, it seemed like low-hanging fruit to me. :)

If minisphere 1.1 doesn't end up being the most awesome Sphere implementation ever after this... well, I don't know what to tell you. :D
Title: Re: minisphere 1.1b1 (stable: 1.0.10)
Post by: Flying Jester on April 23, 2015, 12:53:46 pm

From what I'd expect, all to almost all of the configuration options should be in a built in menu bar of some kind, like when using Visual Boy Advance, so you can change all those options on the fly. It makes a lot more sense to me, but I'm not sure how feasible it would be to change which graphics and audio plugins you're using on the fly.


Hmm....HMMMMM!


Well, I decided on a whim that 1.1 doesn't represent a big enough evolution and am now going to implement the Sapphire API from TurboSphere.  After looking at the method signatures and such, it seemed like low-hanging fruit to me. :)

If minisphere 1.1 doesn't end up being the most awesome Sphere implementation ever after this... well, I don't know what to tell you. :D


I wonder if my map engine would run in minisphere then.
Title: Re: minisphere 1.1b1 (stable: 1.0.10)
Post by: Fat Cerberus on April 23, 2015, 01:17:19 pm


From what I'd expect, all to almost all of the configuration options should be in a built in menu bar of some kind, like when using Visual Boy Advance, so you can change all those options on the fly. It makes a lot more sense to me, but I'm not sure how feasible it would be to change which graphics and audio plugins you're using on the fly.


Hmm....HMMMMM!


Yeah yeah, plugin support, I can take a hint. ::)  I'll get there eventually... maybe.  Honestly it's just not that important to me and adds complexity that's just not needed for most use cases.  Maybe I'll do it when I run out of features to steal from other engines. ;)

As for Sapphire, I just need some clarification: Can a shape be created with a Surface as texture, or does it have to be an Image?  With the way the engine is set up right now I could both (surfaces and images are the same under the hood), but I'm trying to be compliant here.
Title: Re: minisphere 1.1b1 (stable: 1.0.10)
Post by: Flying Jester on April 23, 2015, 01:44:13 pm
In TurboSphere, a Shape must be constructed using an Image.

I wasn't bugging you for plugins...I was just thinking...
TurboSphere could actually unload and load plugins on the fly, but the only issue with that would be that any objects constructed using a plugin that is then unloaded couldn't be used anymore, and when they are finalized bad things would happen...but it's an interesting idea.
Title: Re: minisphere 1.1b1 (stable: 1.0.10)
Post by: Fat Cerberus on April 23, 2015, 01:56:37 pm
I'm assuming UV is normalized [0-1], but what of coordinates?  Are X/Y resolution-dependent or are they normalized too?  And where is the origin (0,0) relative to the screen?

I'm probably going to change a few names here and there in any case (Shape:image->Shape:texture among others), so it won't be a fully compatible implementation, but the general API structure should be identical.
Title: Re: minisphere 1.1b1 (stable: 1.0.10)
Post by: DaVince on April 23, 2015, 03:55:06 pm
Quote
Unless you mean an API to set the mappings, in which case... eh.  I think at the point your game wants to do that, it means you're already exposing your own configuration UI anyway, so might as well go all the way and manage the settings yourself.  Let's not invite scope creep here. :)

That's what I meant, though. Setting options. But yeah, if you're going to have a menu bar it's not going to be needed. :)

Quote
Oh, and speaking of Sir Boingers, I'm actually not a big fan on the new control setup in v2.  The 2010 version's setup was better, I think (Ctrl to bounce).  A big issue with the new controls I've had is that, when bouncing and trying to move left and float at the time, the Up input gets ignored because of simultaneous-key limitations on cheaper keyboards (the keyboard on my Core i3 2-in-1 laptop, for example).  This makes a few areas almost impossible and cost me a good number of avoidable deaths.  I think this is why old console emulators used to use Ctrl and Alt for the face buttons, actually, since modifier keys don't contribute to the 3-key limit.

It's set up so you can use the Z key too. In the end I moved away from ctrl because it felt less natural for modern games, but I can bring it back easily enough.
Title: Re: minisphere 1.1b1 (stable: 1.0.10)
Post by: Fat Cerberus on April 23, 2015, 04:03:34 pm
You can now do this in the latest build of minisphere:

Code: (javascript) [Select]
var image = new Image('texture.png');
var shape = new Shape([
{ x: 0, y: 0, u: 0.0, v: 0.0 },
{ x: 10, y: 0, u: 1.0, v: 0.0 },
{ x: 10, y: 10, u: 1.0, v: 1.0 },
{ x: 0, y: 10, u: 0.0, v: 1.0 }
], image);
var group = new Group([ shape ], GetDefaultShaderProgram());

// vvv this requires access to the Specs Threader, though...
Threads.doWith(group, function() { return true; }, function() { this.draw(); });


Gangnam TurboSphere style!  It draws a little 10x10 textured square at the top left of the screen. ;D

I feel like I should change the order of the arguments for the Shape and Group constructors though... it feels clunky tacking the image/shader onto the end of the array literal like that, it would be more natural as the first argument I think.
Title: Re: minisphere 1.1b1 (stable: 1.0.10)
Post by: Flying Jester on April 23, 2015, 05:03:19 pm
The default shader returned from GetDefaultShaderProgram must guarantee raster coords and have the origin in the upper left, just like how Sphere does it.

UV is normalized, but giving a number greater than 1 causes tiling just like how OpenGL and (or so I've heard) DirectX do it. UV is also automatically assigned to the order you gave if it is not present in the vertices. Triangles use the same coords, but just the first three. Same with lines and points.

I don't think it would be more natural with the Image/Shader first...for the primitives in Sphere, coords and parameters come first, colors and attributes come last. That's why I set it up this way, anyway.

I'll be curious to see how you tackle drawing text using Galileo/Sapphire. That's kind of a thorny spot so far. It works, but I think it could be handled better.
Title: Re: minisphere 1.1b1 (stable: 1.0.10)
Post by: Fat Cerberus on April 23, 2015, 05:10:40 pm

The default shader returned from GetDefaultShaderProgram must guarantee raster coords and have the origin in the upper left, just like how Sphere does it.

UV is normalized, but giving a number greater than 1 causes tiling just like how OpenGL and (or so I've heard) DirectX do it. UV is also automatically assigned to the order you gave if it is not present in the vertices. Triangles use the same coords, but just the first three. Same with lines and points.

I don't think it would be more natural with the Image/Shader first...for the primitives in Sphere, coords and parameters come first, colors and attributes come last. That's why I set it up this way, anyway.


When I said more natural, I meant the syntax.  If a function is likely to be called with a multi-line array or object literal as argument, as here, it's awkward to have to add arguments after the literal.  See the code sample above, especially the Shape initialization, for what I mean.

I'm curious about the automatic UV assignment... how does that work?  Right now UV defaults to zero for all vertices in my implementation, which is pretty useless.  Maybe I should take a closer look at the TS source...

As for that shader, that's a vertex shader, right?  I wasn't entirely clear on whether it was that or a fragment/pixel shader.
Title: Re: minisphere 1.1b1 (stable: 1.0.10)
Post by: Flying Jester on April 23, 2015, 05:46:07 pm

I'm curious about the automatic UV assignment... how does that work?  Right now UV defaults to zero for all vertices in my implementation, which is pretty useless.  Maybe I should take a closer look at the TS source...


If you are curious:
https://github.com/FlyingJester/TurboSphere/blob/master/src/plugins/Sapphire/script.cpp#L608-L679 (https://github.com/FlyingJester/TurboSphere/blob/master/src/plugins/Sapphire/script.cpp#L608-L679)
I check the number of vertices against a lookup table with default UV numbers. As I iterate the vertices, I check if they have U, V, and Color properties, and override the defaults if they do.

That way, to blit an image you don't need explicit U and V, you just need the corner locations. This is pretty convenient, I find. It also allows overriding the color mask, but defaulting to full masking.



As for that shader, that's a vertex shader, right?  I wasn't entirely clear on whether it was that or a fragment/pixel shader.


So, this is what it would look like if TurboSphere had full shader support:

Code: (JavaScript) [Select]


var fragment_shader = new FragmentShader(frag_shader_source_string);
var vertex_shader = new VertexShader(vert_shader_source_string);
var shader = new ShaderProgram(fragment_shader, vertex_shader);

/*...*/
var group = new Group(some_shape_array, shader);


There are other things I could do, too. Maybe add the ability to specify GLSL version in the fragment and vertex constructor and inject that into the shader source? That would be nice if I also added a way to query valid GLSL versions. When the ShaderProgram constructor exists, I will add GetDefaultVertexShader and GetDefaultFragmentShader, so that you can use only a single part of the defaults. Then you'd really need to know the GLSL version.

You need both vertex and fragment shaders in OpenGL 3.2+ and 4. Since that's also valid for 2, that's how I did it. I never got much into shaders for OpenGL 2, maybe it's necessary there to?

Adding Geometry shader passes would also be a very cool extension. I don't really want to get into Compute shaders since that is an aborted feature.

But that's the gist of how I imagine the shader system.
Title: Re: minisphere 1.1b1 (stable: 1.0.10)
Post by: Fat Cerberus on April 23, 2015, 07:56:14 pm
I think Allegro might actually support shaders... I should try experimenting with it. :)

Is Sapphire (I'm assuming Sapphire is the frontend and Galileo the backend...) part of the Pegasus spec by any chance?  It's actually been a while since I've looked at that repo and I don't remember if anything like this was specified in it.
Title: Re: minisphere 1.1b1 (stable: 1.0.10)
Post by: Flying Jester on April 23, 2015, 08:52:58 pm

I think Allegro might actually support shaders... I should try experimenting with it. :)

I wanted shaders to really be optional just because I didn't want the engine to necessarily use OpenGL. GLSL shaders would be an extension to Pegasus.


Is Sapphire (I'm assuming Sapphire is the frontend and Galileo the backend...) part of the Pegasus spec by any chance?  It's actually been a while since I've looked at that repo and I don't remember if anything like this was specified in it.

Sapphire is the name of TurboSphre's implementation (well, the whole TurboSphere graphics plugin). Galileo is the name of the API.

The spec is in the api/ts directory of Pegasus. I just updated a week or so ago, based on what TurboSphere actually implements.
Title: Re: minisphere 1.1b1 (stable: 1.0.10)
Post by: Fat Cerberus on April 23, 2015, 08:59:46 pm
Oh, good to know.  I will rename the extension to "galileo-api" then.  :)
Title: Re: minisphere 1.1b1 (stable: 1.0.10)
Post by: Fat Cerberus on April 23, 2015, 10:57:57 pm
Wow, just took a look at the Galileo spec and it really is a ridiculously simple API.  I imagine it could be deceptively powerful in the right hands, though.

My only real gripe is with the way Vertices are treated.  It specifies a full constructor for what is ultimately just data.  And what's up with Shapes not being able to share a Vertex?  That's just silly. :P
Title: Re: minisphere 1.1b1 (stable: 1.0.10)
Post by: Flying Jester on April 23, 2015, 11:12:34 pm
Shapes are sort of passed by value. In TurboSphere, there isn't even a true vertex constructor, they are just objects. When a Shape is constructed, the data of all the Vertices is copied.

Shapes can 'share' a Vertex, but changes to the Vertex do not affect the Shape.
Title: Re: minisphere 1.1b1 (stable: 1.0.10)
Post by: Fat Cerberus on April 24, 2015, 02:12:39 am
Well, not even 24 hours later, my Galileo implementation is almost finished (sans shaders, of course--that'd be getting a bit too ambitious for 1.1, although Allegro 5.1 *does* have the functionality). :D  It'll be a fully Pegasus compliant implementation and its existence is reported by GetExtensions() as "sphere-galileo".  Allegro 5 truly is awesome.

The only thing I really have left to implement is the automatic U/V assignment and modifiable vertex and shape lists for Shapes and Groups, respectively.  Shouldn't be too much more work.

minisphere 1.1 will be a sight to behold once it's released.  Best of all: You get all this new stuff, and it STILL runs old Sphere games! ;D
Title: Re: minisphere 1.1b1 (stable: 1.0.10)
Post by: Flying Jester on April 24, 2015, 02:30:03 am
That's good. Galileo is meant to be close to how the lower level APIs work in concept, so it's pretty easy to actually implement. Reimplementing it in OpenGL 2.1 only took me a couple hours, most of the work was making it selectable between that and OpenGL 4 and ensuring I had not broken the OpenGL 4 stuff.
Title: Re: minisphere 1.1b1 (stable: 1.0.10)
Post by: Fat Cerberus on April 24, 2015, 10:38:01 am
Just for the hell of it I tried to run TurboSphere's map engine in minisphere, and upon RequireSystemScript'ing turbo/map_engine.js I got this back from Duktape:
Code: [Select]
RangeError: C call stack depth limit


Apparently there's some infinite recursion somewhere and I have no clue why.

Edit Figured it out.  I had minisphere set up to update the RequireScript inclusion tracking info only after executing the script, which is useless in case of mutual inclusion.  Surprising that this bug didn't bite me until now...

Of course, sadly it turns out it doesn't work.  I get a "not callable" error in segment.js line 6.

Hm, interesting.  The issue seems to be that the Turbo object created in each script is localized to that evaluation.  It's not actually being created in the global scope, apparently.  I added an Abort call immediately after the format.js inclusion in segment.js to print out the value of Turbo, and got back 'undefined'.  No idea what's up with that.  If I RequireSystemScript 'turbo/format.js' directly, Turbo is not undefined, so...
Title: Re: minisphere 1.1b1 (stable: 1.0.10)
Post by: Flying Jester on April 24, 2015, 01:15:13 pm
Bear in mind it's possible that TurboSphere's map engine has some stuff in it that doesn't work on a language level in Sphere 1.5 or similar.

There is one other issue to running the map engine. I escape "#~" in paths to be the system directory. I had intended to just put the format JSON files into a local directory to use the map engine with other engines.
Title: Re: minisphere 1.1b1 (stable: 1.0.10)
Post by: Fat Cerberus on April 24, 2015, 05:36:44 pm
It looks like the biggest obstacle to minisphere being able to use the Turbo runtime right now is Duktape's lack of ArrayBuffers.  He's working on it right now (milestone of 1.3.0 listed on the relevant GitHub issue), but it seems to be slow going.
Title: Re: minisphere 1.1b1 (stable: 1.0.10)
Post by: Flying Jester on April 24, 2015, 06:08:24 pm
Can you see where it properly needs one? I tried to allow it to use ByteArrays as well (or any object that has a `[]' operator). I may have left in a couple evil engine calls, though, like making surfaces directly out of ArrayBuffers.
Title: Re: minisphere 1.1b1 (stable: 1.0.10)
Post by: Fat Cerberus on April 24, 2015, 06:47:51 pm

Can you see where it properly needs one? I tried to allow it to use ByteArrays as well (or any object that has a `[]' operator). I may have left in a couple evil engine calls, though, like making surfaces directly out of ArrayBuffers.


I haven't looked particularly hard, but I did find this:
Code: (javascript) [Select]
Turbo.LoadSystemScheme = function(name){

    var scheme_file = new RawFile("#~/formats/"+name);
    var scheme_buffer = scheme_file.read(scheme_file.size);
   
    var scheme_byteview = new Uint8Array(scheme_buffer);
    var scheme_bytearray = ByteArrayFromTypedArray(scheme_byteview);

    return JSON.parse(CreateStringFromByteArray(scheme_bytearray));
}


Which is the most schizophrenic piece of code I've ever seen. :P  You go from a ByteArray (returned by RawFile:read) to a Uint8Array, back to a ByteArray again, and finally to a string before parsing it as JSON.  Were you just stress-testing the ByteArray stuff on purpose or what?

Also, bytearray.js RequireSystemScript's arraybuffer_bytearray.js without checking to see if a native ByteArray already exists.
Title: Re: minisphere 1.1b1 (stable: 1.0.10)
Post by: Flying Jester on April 24, 2015, 07:20:06 pm

Which is the most schizophrenic piece of code I've ever seen. :P  You go from a ByteArray (returned by RawFile:read) to a Uint8Array, back to a ByteArray again, and finally to a string before parsing it as JSON.  Were you just stress-testing the ByteArray stuff on purpose or what?


It's an extremely cheap operation to convert between ByteArrays and TypedArrays in TurboSphere. It just sets a couple properties of the TypedArray to be compatible methods for a ByteArray.

Previously, read() always returned an ArrayBuffer, and ByteArrayFromTypedArray has wavered between only handling TypedArrays and ArrayBuffers. Now it's able to handle either.
So you're completely right, that code is just too much, making up for issues that don't even exist anymore.

Edit:
Ah, I see. The whole file reader system needs a file reader for ByteArrays.
Title: Re: minisphere 1.1b1 (stable: 1.0.10)
Post by: Fat Cerberus on April 25, 2015, 10:39:47 am
Last night I did a massive reorganization of the minisphere directory.  It's now set up like a real cross-platform C app instead of the MSVC-generated structure it had before.  Source and headers are in 'src', the MSVC project files are in 'msvc', and the engine is built in 'bin'.  A small thing, but at least now it doesn't look like cross-platform support is an afterthought. ;)

@DaVince and @casiotone: Could you guys check that it still builds on Linux and OS X using SCons?  I updated the build scripts to reflect the new structure but as SCons won't build it for me on Windows, I can't test that everything is working properly.
Title: Re: minisphere 1.1b1 (stable: 1.0.10)
Post by: DaVince on April 25, 2015, 12:12:26 pm
Yep, still builds here. I encounter a segfault when dying in Sir Boingers though.
Code: [Select]
fish: Job 1, "./msphere ~/Dropbox/projects/sphere/boingers2.2/startup/" terminated by signal SIGSEGV (Address boundary error)

(I use fish instead of bash so the segfault message looks a bit different.)
Title: Re: minisphere 1.1b1 (stable: 1.0.10)
Post by: Fat Cerberus on April 25, 2015, 12:22:56 pm
Is the segfault reproducible?  I know I encountered a few myself with recent builds, but they were always in the Vorbis code so I didn't know what to do about it.
Title: Re: minisphere 1.1b1 (stable: 1.0.10)
Post by: DaVince on April 25, 2015, 12:34:42 pm
Happens every time I kill myself.

I think it chokes on the dying screen, as my new version uses the original IT files now and this is one that should play only once. I can hear about 0.2 seconds of the music before it segfaults.
Curiously, the win screen does function, even though it also calls a non-looping IT file... (It loops anyway which isn't right though.)

I'll send you the version I'm messing around with.
Title: Re: minisphere 1.1b1 (stable: 1.0.10)
Post by: DaVince on April 25, 2015, 12:42:32 pm
I also have a request: could you make minisphere check if there's only one game in the games folder and just run it if that's the case? I don't want to put my game into a startup folder because that makes it more of a hassle to package the game without the engine.
Title: Re: minisphere 1.1b1 (stable: 1.0.10)
Post by: Fat Cerberus on April 25, 2015, 12:50:01 pm
Yeah, the looping thing is a bug in Allegro that's fixed in the latest build but hasn't been released yet.  There's nothing I can do on minisphere's end to work around it, unfortunately.  It does work properly on Windows because I compiled Allegro myself and included the appropriate fix. :). I actually had no choice, because there are no pre-built x64 MSVC binaries for Allegro, or at least I haven't found any...
Title: Re: minisphere 1.1b1 (stable: 1.0.10)
Post by: DaVince on April 25, 2015, 12:52:20 pm
Yeah, I noticed it works fine on Wine. Will wait for the next Allegro version then.

I also noticed that in Windows the file selector defaults to My Documents instead of the working directory. Might want to make the default path either minisphere's directory, or the games subdirectory (when found). :)
Title: Re: minisphere 1.1b1 (stable: 1.0.10)
Post by: Fat Cerberus on April 25, 2015, 12:58:55 pm

Yeah, I noticed it works fine on Wine. Will wait for the next Allegro version then. :)


Now that you mention it, I remember that there was another bug that was fixed along with the looping: a buffer overrun in the module streaming code. So that's probably where your segfault is. :)

...and in case there was any doubt, I'm the one that discovered it. ;)
Title: Re: minisphere 1.1b1 (stable: 1.0.10)
Post by: DaVince on April 25, 2015, 01:26:07 pm
Yep, I remember when it all transpired. Just thought that the included version of Allegro was your fixed version, so I thought "hm, maybe it's not fixed after all". Until you clarified that just now.
Title: Re: minisphere 1.1b1 (stable: 1.0.10)
Post by: Fat Cerberus on April 27, 2015, 12:27:37 am
I've decided what I want to do for the configuration editor.  The config tool will be written in JS and run by minisphere as any other Sphere game.  config.exe will be a tiny stub that simply defers to the engine and points it at the location of the config "game" game.sgm.

Unfortunately this does mean that key mappings won't be able to be changed while in-game, but after a short-lived experiment trying to implement a graphical config on the native side (the only way to implement such a thing without interrupting the currently running game), I gave up.  C--or any purely procedural langauge, really--just isn't suited for writing a GUI.  It'd much less painful in an OO language like JavaScript.  Besides, as with Sphere's old config.exe, it's really just meant to be a convenience; the preferred option will always be to implement your own config in-game to ensure UI consistency and whatnot.

On a sidenote, my other attempt at a config tool was in C++ using wxWidgets for a Sphere 1.5-style tool, but 1) wxWidgets is horrendously bloated (it took me 10-15 minutes to compile all the libraries on a 5th-gen Core i3--note that this is a CPU that can run the Dolphin emulator!) and 2) It very quickly reminded me why I hate C++.  So yeah, that was a failure too.
Title: Re: minisphere 1.1b1 (stable: 1.0.10)
Post by: Radnen on April 27, 2015, 12:40:39 am
So it has become like the startup game, but also like a library that you can add to your game. Neat!
Title: Re: minisphere 1.1b1 (stable: 1.0.10)
Post by: Flying Jester on April 27, 2015, 12:41:02 am
Just my opinion, having had a long time to wonder about what GUI toolkits to use...

You should take a look at fltk. It is kind of intended for what you describe. It's extremely lightweight, and it makes only a very select usage of C++'s features. It is also extremely cross platform, and still provides easy integration with platform tools.

For instance, you could easily use Fl_x.h's stuff to create a new parent window given a Win32, X11, or Cocoa window handle to have as a child. I was considering using that to frame the window SDL2 creates for TurboSphere, so that I could have an SDL2 OpenGL window embedded in an fltk window that has a menu bar for this kind of thing. You could also automatically use the OS X menu bar instead of a separate one by using Fl_Sys_Menu_Bar.

I despise Qt and WxWidgets because they are ridiculously large and solve simple problems in the most complex way imaginable. I don't like GTK because its maintainers have no commitment to backwards compatibility at all. I rather like fltk because it doesn't have any of these issues. It can look a little dated with the default scheme, but that's really the least of what matters to me. The theme is very usable and simple.
Title: Re: minisphere 1.1b1 (stable: 1.0.10)
Post by: Fat Cerberus on April 27, 2015, 01:21:04 am
I do like the GTK+ theme, if I have to use non-native widgets that's the one I'd prefer by far.  But GTK is pretty bloated itself.  The redistributable last I remember was about 20MB and there's no way to static link to it.  No way I'm carrying around that kind of baggage for a config tool that half of Sphere users aren't even going to touch.  I did take a look at fltk, and while I think most of those screenshots are pretty fugly, it does look very lightweight and simple to use.

But I really think the startup-game-style JS config is the way to go here.  This probably sounds odd given my dedication to doing everything in native, but at the end of the day I don't consider this core functionality; it's just a nice extra.  This way if a game doesn't need it--say, because it implements its own configuration menus--config.exe and the config folder can be left out of the distro.  It's a different situation than, say, the map engine, where at least 3/4 of all Sphere games utilize its functionality in some way and so it's beneficial to have it built into the engine. (And I wouldn't have wanted to have to debug that thing in JS anyway--the MSVC debugger was invaluable during its development).
Title: Re: minisphere 1.1b1 (stable: 1.0.10)
Post by: Flying Jester on April 27, 2015, 01:24:42 am

And I wouldn't have wanted to have to debug that thing in JS anyway--the MSVC debugger was invaluable during its development


Fortunately, both SpiderMonkey and V8 can be run so that gdb or lldb can view JS stack frames as though they were native frames :)
Or at least V8 used to be able to. When I was on my way out of the V8 world, it had been broken for some time and there were murmurs of that being removed.

But I found that one major advantage of the map engine in script was simply that it is easier to write something purely behavioural like that in JS. It needs much less debugging because it has fewer bugs in JS.
Title: Re: minisphere 1.1b1 (stable: 1.0.10)
Post by: Radnen on April 27, 2015, 01:52:07 am

But I found that one major advantage of the map engine in script was simply that it is easier to write something purely behavioural like that in JS. It needs much less debugging because it has fewer bugs in JS.


It's also easily adaptable by the community. ;)
Title: Re: minisphere 1.1b1 (stable: 1.0.10)
Post by: Fat Cerberus on April 27, 2015, 01:57:34 am


But I found that one major advantage of the map engine in script was simply that it is easier to write something purely behavioural like that in JS. It needs much less debugging because it has fewer bugs in JS.


It's also easily adaptable by the community. ;)


That's true, but also remember that, as discussed in this very thread, due to subtle, easy-to-take-for-granted design choices (such as that infamous edge script triggering double-snafu I hit) it's already pretty extensible as-is.  Your Blockman game is proof of that. :D  And Lithonite...
Title: Re: minisphere 1.1b1 (stable: 1.0.10)
Post by: Radnen on April 27, 2015, 02:59:26 am

That's true, but also remember that, as discussed in this very thread, due to subtle, easy-to-take-for-granted design choices (such as that infamous edge script triggering double-snafu I hit) it's already pretty extensible as-is.  Your Blockman game is proof of that. :D  And Lithonite...


Well that's because of SetUpdateScript and SetRenderScript. Basically it let's you "have at it", that coupled with the copious number of getters and setter available to you, there's little that can stand in your way. That said it's still not 100% flexible and even Lithonite would have been easier to code / maintain if it knew to hook into certain areas of the engine easier. There's a lot of hack code too, as a result of Sphere's map API. But it is rather open.

I mean, most maps are indeed very simple structures. You have tiles, layers, entities, zones & triggers. Some objects have obstruction, some tiles have animations, some layers move or parallax. It really covers everything.

That said there currently is no way to exclusively draw a map wherever you want. It'd be nice to treat maps like a standalone object, such that you can muck with it's properties after loading, and draw portions of you choice of the map, wherever, whenever. That's quite hard in Sphere since you are limited to one map per screen. There are double camera tricks, but it can only happen on the same map... unless you write your own map loading call.

I want an engine where I can load up and string maps end to end, with some dynamic calls. Allocate some maps, and deallocate those of which that are out of screen, and then reload them when you are near again. I used this tactic in other game engines to create 'infinite' maps. Maps without borders, but are loaded by stringing maps end-to-end dynamically. The performance benefit is obvious here since you aren't loading a 999x999 sized map all at once.
Title: Re: minisphere 1.1b1 (stable: 1.0.10)
Post by: DaVince on April 27, 2015, 05:57:18 am
If the config tool is contained within a single script, game devs could opt to run it within the game itself too, like

Code: (javascript) [Select]
RequireScript("sphere-config.js");
SphereConfigMenu();


And even customize it to their own likings if they really wanted. :)
Title: Re: minisphere 1.1b1 (stable: 1.0.10)
Post by: Fat Cerberus on April 27, 2015, 09:28:00 am

If the config tool is contained within a single script, game devs could opt to run it within the game itself too, like

Code: (javascript) [Select]
RequireScript("sphere-config.js");
SphereConfigMenu();


And even customize it to their own likings if they really wanted. :)


That's actually a really good idea, I'll have to include the config core as a system script now. :). I was thinking it would look something like the Xrossbar menu on the PS3, with an optional game-specific background taken from the game.sgm root, similar to how icon.png is now.  This way if a dev chooses not to make their own config, the default one won't completely clash with whatever they make.
Title: Re: minisphere 1.1b1 (stable: 1.0.10)
Post by: Fat Cerberus on April 27, 2015, 12:20:08 pm

That said there currently is no way to exclusively draw a map wherever you want. It'd be nice to treat maps like a standalone object, such that you can muck with it's properties after loading, and draw portions of you choice of the map, wherever, whenever. That's quite hard in Sphere since you are limited to one map per screen. There are double camera tricks, but it can only happen on the same map... unless you write your own map loading call.

I want an engine where I can load up and string maps end to end, with some dynamic calls. Allocate some maps, and deallocate those of which that are out of screen, and then reload them when you are near again. I used this tactic in other game engines to create 'infinite' maps. Maps without borders, but are loaded by stringing maps end-to-end dynamically. The performance benefit is obvious here since you aren't loading a 999x999 sized map all at once.


See, if I were writing an engine like Sphere from scratch (i.e. without a base to work from), I almost certainly would have done the map engine in script and included it as a system script (I probably wouldn't have chosen JS though, Lua would have been my first choice I think).  But, knowing what I do now, I'd want a default map engine of some sort built into the engine.  An additional RequireScript() doesn't sound like much to you or me, but let me just say from experience, it's a big deal for someone just learning Sphere.  The first time I used Require/EvaluateScript in a game was something of a rite of passage; it was like I was graduating from "Sphere novice" to "advanced Sphere guy". :P  Don't underestimate the value of being able to do a ton of stuff without importing a single external script--it's empowering and is, in large part, what's made Sphere so approachable over the years.

Moving on: Regarding your ideas, let me just tell you a story.  Everyone thinks they want an object-oriented map engine, but I actually wrote one years ago in C++ and I hated using it.  NPC management was a pain in the ass, and just in general, putting together a working game required a bunch of tedious micromanagement of entity objects.  This wasn't a fault of the map engine itself--it was quite powerful--but simply that encapsulation isn't well suited to managing a map engine with all its various interconnected components.  In minisphere for instance there is very much a circular dependency between the map engine and persons manager that makes it hard to fully componentize.

What it basically comes down to is that OOP relies on the assumption that, as long as someone, somewhere, holds a reference to an object, that object remains valid.  So now, assume for the sake of argument that you do this:
Code: (javascript) [Select]

var townMap = new Map('MarketTown.rmp');
if (/* some condition that triggers this guy being there */) {
  var traveler = new Person("Some Guy", "TravelingSalesman.rss", map);
  // or maybe: townMap.createPerson(...)
}
// some more code and somewhere along the way, a reference to the traveler object is stored
// and persists past map destruction


The problem is, if `townMap` gets unloaded somewhere along the way, `traveler` is implicitly disposed and it's no longer safe to use any references to it.  Even though the object that holds the reference may have no way of knowing that.  This is why I like Sphere's system of decoupling persons from maps and referencing them by name: If all you have is a name, there are no assumptions that the person exists at any given moment, and it's then natural to call DoesPersonExist() first to check.  `person.doesExist()` is more awkward is totally against the OOP philosophy: The fact I have a reference at all and haven't explicitly disposed it should automatically mean it exists.

The alternative, of course, is to make Person objects completely independent from Map objects.  But at that point we're essentially back where we started, just with some fancy constructors tacked on.  And it's *still* not safe for the map engine to automatically destroy them since even in this case, object references are being passed around.

Being able to load and chain together multiple maps via independent objects would be nice, of course, I'm just not sure it's worth all the headaches.
Title: Re: minisphere 1.1b1 (stable: 1.0.10)
Post by: Fat Cerberus on April 27, 2015, 03:37:20 pm
@DaVince: I added some console output to the latest builds, including what you suggested, printing out the error message (and file/line) for uncaught JS exceptions.  It also logs loading and unloading sounds, which I'm not sure whether that's being too verbose or not...

Mainly I'm just afraid that too many printf's will be a performance issue.
Title: Re: minisphere 1.1b1 (stable: 1.0.10)
Post by: Flying Jester on April 27, 2015, 04:42:40 pm

@DaVince: I added some console output to the latest builds, including what you suggested, printing out the error message (and file/line) for uncaught JS exceptions.  It also logs loading and unloading sounds, which I'm not sure whether that's being too verbose or not...

Mainly I'm just afraid that too many printf's will be a performance issue.


It really can be on Windows. The console is silly slow there. Everywhere else, the slow part is the format string parsing.
Title: Re: minisphere 1.1b1 (stable: 1.0.10)
Post by: Radnen on April 27, 2015, 09:18:39 pm
LordEnglish: Hey, all I can say is OOP maps is a solved problem, the way we deal with entities may have to change. That said Sphere's API really does simplify a lot, but it will always be an amateur API in my mind.

That said even if you don't go OOP on maps you can still have a system where you load up a map and draw potions of it on screen, because I know that is a hard problem to solve in Sphere presently.

Code: (javascript) [Select]

// basically you can only call this inside a MapEngine context
// it loads the map, puts it's characters into the same entity batch as the map engine,
// then draws it's tiles and obstructions starting from the specified offset (in this case 640x0)
// functions like DoesPersonExist are still preserved and you still can't make assumptions on the existence of entities.
var map = LoadMap("map.rmp", 640, 0);


The camera is the hardest part to figure out, because now we are giving freedom as to where you put the map. It's easier in fact to tell Sphere to instead, put the map on the edge of another map, moving or removing the edge script and making the camera realize there is in fact more to scroll to.

Code: (javascript) [Select]

// what it looks like using positional constants, which is very much a Sphere-like thing to do:
var map = LoadMap("map.rmp", MAP_EAST);

Title: Re: minisphere 1.1b1 (stable: 1.0.10)
Post by: Fat Cerberus on April 27, 2015, 10:12:22 pm
Now that I think about it some more, splicing maps together like this actually seems like a hack.  It's essentially an optimization to avoid loading the whole map into memory at once, right?  Which makes sense, but wouldn't it be better--and simpler--to solve the problem in the map engine itself (whether in script or native, OO or otherwise), where if you have a huge map, the engine can automatically load and unload pieces of it on demand, just like how, when you have a huge, multi-gigabyte file, opening it in hex editor is instantaneous because it only loads a few KB at a time.

Essentially we'd be letting the map engine "stream" the map, while still allowing you to work on it as one huge world in the editor.
Title: Re: minisphere 1.1b1 (stable: 1.0.10)
Post by: Flying Jester on April 27, 2015, 11:01:54 pm
It's not all that difficult to make a basic OOP map engine that is fairly simple to use. I did it, or at least got the rough outline of it as the core of the TurboSphere map engine. It's all object based, and it works pretty easily.

I did make persons belong strongly to maps. It's not that bad a thing to do, it works fairly well.

You don't need to make a doesExist method. You just need to maintain an array of persons for the map. If the person is not in that array, they aren't on that map.

But I also believe that the map engine does not belong as a native, core part of the engine. It's one thing to include one, but it's not a core function. Chad Austin came to this realization before Sphere 1.0 was even released. He decided it would be better to expose a simple but well thought out scene graph and scene manager rather than a rigid map engine.

Both those components are a part of Pegasus. I disagree with some others on the necessity of the scene manager. I think it's fine for this:

Code: [Select]

function game(){
    while(true){}
}


to hang the engine. If we subscribe to a Mozilla-style view of JavaScript, where it is the C or Assembly of the web or our games, we need to trust it with tasks we trust C with. Even like managing the event loop.
Title: Re: minisphere 1.1b1 (stable: 1.0.10)
Post by: Fat Cerberus on April 27, 2015, 11:44:20 pm

It's not all that difficult to make a basic OOP map engine that is fairly simple to use. I did it, or at least got the rough outline of it as the core of the TurboSphere map engine. It's all object based, and it works pretty easily.

I did make persons belong strongly to maps. It's not that bad a thing to do, it works fairly well.

You don't need to make a doesExist method. You just need to maintain an array of persons for the map. If the person is not in that array, they aren't on that map.

But I also believe that the map engine does not belong as a native, core part of the engine. It's one thing to include one, but it's not a core function. Chad Austin came to this realization before Sphere 1.0 was even released. He decided it would be better to expose a simple but well thought out scene graph and scene manager rather than a rigid map engine.

Both those components are a part of Pegasus. I disagree with some others on the necessity of the scene manager. I think it's fine for this:

Code: [Select]

function game(){
    while(true){}
}


to hang the engine. If we subscribe to a Mozilla-style view of JavaScript, where it is the C or Assembly of the web or our games, we need to trust it with tasks we trust C with. Even like managing the event loop.


Totally agree on that last part.

As for the object-oriented maps, let's consider a scenario:
* As you describe, there is an array of entities that each map exposes.
* Someone takes a reference to an entity from that array, and it gets passed around to a bunch of functions.
* The entity reference is ultimately saved somewhere for later use, despite its volatility.
* The map the entity came from gets unloaded. The entity goes with it.
* The object holding the reference has no idea the entity has disappeared and attempts to call methods on it.
* Game goes kaboom due to unexpected, and therefore uncaught, exception ("entity has been destroyed").

This can of course be dealt with by someone knowledgeable in OOP (don't pass around borrowed references), but let's not fool ourselves into thinking a good deal more vigilance is required.  I myself have made mistakes like what I described above.  This is the kind of thing that's likely to scare off novices, which is something we definitely DON'T want.

I don't know, I'm starting to feel like I have a totally different view of "what Sphere is" compared to everyone else here.  Either that or I'm just better at remembering what it was like to be a novice...  Either way, I'm leaving the Sphere 1.x map engine as part of minisphere.  The entire engine is less than 2MB anyway, so it can hardly be considered bloat.  Ultimately what it comes down to is that I'm a fan of "useful defaults": If someone wants to step outside the box that's fine, but sometimes it helps to know where the box is in the first place. ;)

Maybe some day I'll write a replacement map engine for Sphere in JS.  But I'll be perfectly honest, I haven't yet felt the need to do so.

As for the stage manager: I honestly have no clue what the thing is.  I saw it when I was perusing the Pegasus repo and even read the code, but I still have no idea what it's supposed to do.
Title: Re: minisphere 1.1b1 (stable: 1.0.10)
Post by: Flying Jester on April 28, 2015, 12:01:33 am
I'm not saying that I don't think we should include a default map engine. Just that, for TurboSphere, I want it to be implemented in terms of the engine's API, and not exist as a direct part of that API.



As for the object-oriented maps, let's consider a scenario:
* As you describe, there is an array of entities that each map exposes.
* Someone takes a reference to an entity from that array, and it gets passed around to a bunch of functions.
* The entity reference is ultimately saved somewhere for later use, despite its volatility.
* The map the entity came from gets unloaded. The entity goes with it.
* The object holding the reference has no idea the entity has disappeared and attempts to call methods on it.
* Game goes kaboom due to unexpected, and therefore uncaught, exception ("entity has been destroyed").

This can of course be dealt with by someone knowledgeable in OOP (don't pass around borrowed references), but let's not fool ourselves into thinking a good deal more vigilance is required.  I myself have made mistakes like what I described above.  This is the kind of thing that's likely to scare off novices, which is something we definitely DON'T want.



I guess to me I don't really see why this would be all that different than if you used the name of a person that had been destroyed in a previous map in the 1.5 API.
Title: Re: minisphere 1.1b1 (stable: 1.0.10)
Post by: Fat Cerberus on April 28, 2015, 12:15:53 am

I guess to me I don't really see why this would be all that different than if you used the name of a person that had been destroyed in a previous map in the 1.5 API.


It might just be my pragmatism kicking in.  If all I have is a primitive type (string, integer, whatnot) for a reference, I fully expect the thing it references to be pulled out from under me and I'm more likely to remember to check for existence at critical moments as a result.  Object references I treat as essentially being the object they reference, so I don't expect it to suddenly disappear without my knowing about it.

Maybe I'm just weird. :P
Title: Re: minisphere 1.1b1 (stable: 1.0.10)
Post by: Flying Jester on April 28, 2015, 12:32:23 am
In that respect, though, the person does still exist. If they are no longer attached to a map, they could be placed back on a map, and it would work fine. It's just that you can't be sure if they are on a map unless you know what map they should be on. Since there can be a lot more to a person than what the faux-constructor exposes, this is a chief strength of this approach.

But in this scenario, you would be expressly drawing and updating certain maps. So you do know which maps are active and when, and you could find out pretty easily if the person was a part of an active map.

In my implementation, they could also (probably to some bad effect) be in multiple maps at once. But I haven't considered the issue of really drawing and updating multiple maps yet, other than seeing it's possible. Perhaps at that point, recursive map/person objects would be better? Or to only have a person reference their map and not the other way around? I have no idea.

Doing it as you do, making persons not attached directly to maps works better with this, then, since there is just a single list of persons that are active and updated with maps. If you were to reference the person's owning map from the person, too, that would (I suspect) be a simple way to support multiple simultaneous maps.
Title: Re: minisphere 1.1b1 (stable: 1.0.10)
Post by: Radnen on April 28, 2015, 02:02:13 am

Now that I think about it some more, splicing maps together like this actually seems like a hack.  It's essentially an optimization to avoid loading the whole map into memory at once, right?  Which makes sense, but wouldn't it be better--and simpler--to solve the problem in the map engine itself (whether in script or native, OO or otherwise), where if you have a huge map, the engine can automatically load and unload pieces of it on demand, just like how, when you have a huge, multi-gigabyte file, opening it in hex editor is instantaneous because it only loads a few KB at a time.


I know what you mean, my Sphere Studio does that fast loading (well, technically it stores the integers into memory but that's far cheaper than the grid of tiles. I create these buffer zones, totally inspired by Google Map and I draw those on screen only when they come into view, but they always keep a weak reference to the tiles that belong to them).

That said, yes I am in support of better map streaming, but I still think that you are getting messy with sprites on far corners of the map, and managing huge obstruction grids, but to be perfectly honest I don't know how bad the performance drain really is. I just wonder if you can reference an entity by name and they better exist, even if they hadn't been loaded in yet since technically they are on the far side of the map and not yet "streamed in". This is why performance might be na issue which is why I wanted to go with the cell-like structure that even 3D games like Morrowind used when loading neighboring maps. With the programmer managing the cells there's no illusion to their existence, you'd know if you streamed them in.

I mean Morrowind had to deal with the same issues: referencing an entity on the far end of a map and then waiting until they suddenly deallocate. I've seen this happen and maybe it's a partial cause to the Crash to Desktop people witness. But most of the time even though the reference to the object goes away most classes seem to handle it just fine (damage does nothing, arrows pass through them, minimap indicators disappear, etc.) but then again you may also be right that it could be due to discipline.

And you are right again in that Sphere should be easy to learn. But many things are easy to learn and difficult to master (take StarCraft for instance). I think we can add a more advanced OO API in addition to a functional API like we have, that's all.

In fact, I have made an OO implementation of Sphere person's by wrapping all the get/set method.

[gist=563ba8a4bc6c199877a9][/gist]

I'll say the gist feature is my #1 favorite feature of these forums.

I'll also add that since entities are names I use a cache table in addition to the above so I can always reuse the API without creating more classes (though in reality this is very lightweight).
Title: Re: minisphere 1.1b1 (stable: 1.0.10)
Post by: Fat Cerberus on April 28, 2015, 02:10:01 am
That's why I think the streaming should be handled in the map engine itself rather than the user splicing them together manually from outside.  The map engine is in a much better position to handle difficult cases such as you describe (entities outside the current "load window", etc.) as it has more complete information.  Anything done from outside involves a lot more guesswork.

That said, the Sphere map engine is very lightweight: a map is essentially just an array of integers, a tileset (usually not a huge image) and some entity data.  Even a 1000x1000 map is about 4MB worth of tile indices per layer and a single tileset.  There's basically no benefit to streaming it from disk when you have multiple gigabytes of ram to work with.
Title: Re: minisphere 1.1b1 (stable: 1.0.10)
Post by: Radnen on April 28, 2015, 02:27:32 am

That said, the Sphere map engine is very lightweight: a map is essentially just an array of integers, a tileset (usually not a huge image) and some entity data.  Even a 1000x1000 map is about 4MB worth of tile indices and a single tileset.  There's basically no benefit to streaming it from disk when you have multiple gigabytes of ram to work with.


Yes I know it's no much but I guess I'm more concerned that a bigger map means someone out there will attempt to update 3000 entities simultaneously. But then I guess that's on them then. ;)

Then the only thing I'll add by saying I liked the cell approach better than streaming it all in is because you can grow your maps infinitely in any direction. With one large static map, you kind of pigeon-hole yourself into that size. But I guess maps can always be grown to the right and bottom if needed.
Title: Re: minisphere 1.1b1 (stable: 1.0.10)
Post by: Fat Cerberus on April 28, 2015, 02:34:06 am
The main advantage to the cell approach that I can see is its potential for dynamism: splicing in a different map depending on different factors.  SMB1 World 7-4 type stuff.  So it's not entirely without merit. :D
Title: Re: minisphere 1.1b1 (stable: 1.0.10)
Post by: Radnen on April 28, 2015, 02:38:25 am
Celled maps also give a team project benefit, I mean try merging someone's 1000*1000 sized map with your edits. This can be facilitated with a merge tool, but it can be tricky to pull off.
Title: Re: minisphere 1.1b1 (stable: 1.0.10)
Post by: Fat Cerberus on April 28, 2015, 11:46:16 am
@DaVince Just tried the latest version of Sir Boingers that you sent me via PM, I see you upped the framerate to 60.  This made the gameplay very smooth, but apparently had a subtle effect on the jump physics that makes it harder to control.  But no crashes.  So yeah, it must just be that you have a stock version of Allegro on Linux that still has the buffer overflow bug.  Apparently Allegro 5.1.10 is coming out soon, though. :)
Title: Re: minisphere 1.1b1 (stable: 1.0.10)
Post by: DaVince on April 28, 2015, 01:12:36 pm
Good to know. And yeah, the physics are a thing to be fixed, the first thing I needed to fix was setting a sane framerate though. :P
Title: Re: minisphere 1.1b1 (stable: 1.0.10)
Post by: Fat Cerberus on April 28, 2015, 01:45:55 pm
What would be a good thing in the engine to be able to configure?  Right now all I can think of is the key/button bindings, which hardly seems worth the effort.  I need more stuff to be able to work up the ambition to do it. :P
Title: Re: minisphere 1.1b1 (stable: 1.0.10)
Post by: DaVince on April 28, 2015, 04:54:26 pm
Fullscreen mode and screen multiplication size would be nice. Also video filter, if you ever implement something like that anyway.
Title: Re: minisphere 1.1b1 (stable: 1.0.10)
Post by: Fat Cerberus on April 29, 2015, 02:08:38 am
It's not a config tool, but... I just implemented automatic U/V coordinate assignment for Shapes, and it's working great. ;D

@FlyingJester: Want to see a neat trick?  I assign default U/V coords like this:
Code: (c) [Select]
bounds = get_shape_bounds(shape);
width = bounds.x2 - bounds.x1;
height = bounds.y2 - bounds.y1;
for (i = 0; i < shape->num_vertices; ++i) {
delta_x = shape->vertices[i].x - bounds.x1;
delta_y = shape->vertices[i].y - bounds.y1;
shape->vertices[i].u = width > 0 ? delta_x / width : 0.0;
shape->vertices[i].v = height > 0 ? delta_y / height : 1.0;
}


Essentially I calculate the shape's bounding box, and then do a bit of linear interpolation on the vertices to get the U/V values.  This works for any number of vertices with no predefined lookup tables needed. :)

There is a downside, of course: If you accept the defaults, the texture basically becomes a decal.  This is the one upside of TS's method: If your 4-cornered shape is something other than a rectangle (a rhombus, say), the texture map remains rectangular (0,0)-(1,1) and the image is properly distorted.

Edit: So, new method, still works for any number of vertices but allows the texture to distort (note: assumes clockwise winding starting from top left):
Code: (c) [Select]
static void
assign_default_uv(shape_t* shape)
{
double phi;

int i;

for (i = 0; i < shape->num_vertices; ++i) {
phi = 2 * M_PI * i / shape->num_vertices - M_PI_4 * 3; // counterclockwise offset 135 degrees
shape->vertices[i].u = (cos(phi) * M_SQRT2 + 1.0) / 2.0;
shape->vertices[i].v = (sin(phi) * M_SQRT2 + 1.0) / 2.0;
}
}


Essentially what I'm doing is circumscribing the UV space--which conveniently happens to be a unit square!--and then setting UV to points on that circle.

Edit2: Correction, clockwise winding.  Trig functions are canonically CCW, but the inverted Y axis reverses the winding.
Title: Re: minisphere 1.1b2 (stable: 1.0.10)
Post by: Fat Cerberus on April 30, 2015, 02:26:24 pm
New beta posted. ;D
Title: Re: minisphere 1.1b2 (stable: 1.0.10)
Post by: Fat Cerberus on May 01, 2015, 01:40:07 pm
I'm presently in the process of updating the Specs Threader.  I'm going to include with minisphere 1.1 as a system script. 8)

It's like an OO version of hook-list.js:
Code: (javascript) [Select]
Threads.initialize();
// ...
var tid = Threads.create(obj);
// obj has .update() and optional .render() called once per frame


It's also a bit like pthreads in that you can do this:
Code: (javascript) [Select]
var worker = Threads.create(obj);
// do some stuff, and then later, perhaps during an update:
Threads.join(worker); // block until worker terminates

...and it will only block the current thread (if any).

For what it's worth I think it might be a good idea to implement some sort of NuGet-type system, where external dependencies such as Link and this can be declared in a the game's metadata and be automatically retrieved.  That's a project for another time, though.
Title: Re: minisphere 1.1b2 (stable: 1.0.10)
Post by: Radnen on May 01, 2015, 08:10:58 pm

For what it's worth I think it might be a good idea to implement some sort of NuGet-type system, where external dependencies such as Link and this can be declared in a the game's metadata and be automatically retrieved.  That's a project for another time, though.


http://forums.spheredev.org/index.php/topic,1262.msg5934.html (like bower)
Title: Re: minisphere 1.1b2 (stable: 1.0.10)
Post by: Fat Cerberus on May 01, 2015, 08:15:52 pm
Is t that just about a way to load modules that already exist in the engine's search path, whatever that happens to be?  I guess it could be extended to search the Internet or something, but I didn't get the feeling that was what it was about.  That's more about modularizing stuff to avoid name clashes.
Title: Re: minisphere 1.1b2 (stable: 1.0.10)
Post by: Radnen on May 01, 2015, 08:48:28 pm

Is t that just about a way to load modules that already exist in the engine's search path, whatever that happens to be?  I guess it could be extended to search the Internet or something, but I didn't get the feeling that was what it was about.  That's more about modularizing stuff to avoid name clashes.


Well, it's kinda both. Dependency management, name clashes and a tool like bower that you can use to pull dependencies automatically from a server (which is what NuGet is like, or NPM).

But this is more like an editor feature if anything. Unless you want to go the node route and inherently work with modules (meaning we must change the way in which we program and expose the code to all of our games going forward).
Title: Re: minisphere 1.1b2 (stable: 1.0.10)
Post by: Fat Cerberus on May 01, 2015, 08:52:25 pm
Right, which is what the Pegasus spec currently advocates.  Everything is a module, even built-in engine functions.  Personally I hate that model myself, though I can see the appeal.
Title: Re: minisphere 1.1b2 (stable: 1.0.10)
Post by: Flying Jester on May 02, 2015, 12:20:16 am
One aspect that I find appealing is that, under the plugin model of TurboSphere, you can run dedicated servers for networked games. In the case of TS, you can actually lose the libraries you don't want. The distro becomes smaller. Of course, the flip side is that it is a more complex setup in the first place.

Being able to remove any portion of the engine is important to me. Should someone find the audio plugin unsuitable, for instance, they could write their own or use an alternative, but not lose the other plugins. A Linux model, as opposed to a BSD model, if you will.

But of course, this is an aspect that is rather different in philosophy than Sphere 1.5. It just so happens you can recreate a Sphere-like environment while still working this way.
Title: Re: minisphere 1.1b2 (stable: 1.0.10)
Post by: Fat Cerberus on May 02, 2015, 12:29:39 am
True, I just don't feel, with minisphere being ~1.75MB for a complete 64-bit Sphere 1.x implementation (okay, so some stuff is missing like the Animation API, sue me :P) that works out of the box, plus all the extras it offers, it's worth adding the overhead of plugins (this is why I preferred the monolithic Allegro build before I switched to static linking--use more than about 2 Allegro components and the monolith DLL is actually smaller than the individual ones).

With TurboSphere the plugin model makes more sense, since it's designed for maximum flexibility.  minisphere I see more as a "download it and have a ball" deal.

By the way you might want to update the note about minisphere in the TS readme--I don't think minisphere can be considered in "early development" anymore. ;)
Title: Re: minisphere 1.1b2 (stable: 1.0.10)
Post by: Fat Cerberus on May 02, 2015, 01:20:57 am
I give to you: minithreads!  This is a generic adaptation of the threader I'm using in my game Spectacles: Bruce's Story.  It only works out of the box in minisphere, but would probably work in Sphere 1.5 with some tweaking.

[gist=0d2a95dcbbccf9a5ebb1][/gist]

Usage:
Code: (javascript) [Select]
// note: the below calls SetUpdateScript(); and SetRenderScript() intrinsically:
Threads.initialize();

// if `obj` is an object with .update() and .render() methods:
var threadID = Threads.create(obj);

// or perhaps to encapsulate a threaded thing:
this.threadID = Threads.create(this); // on construction
Threads.kill(this.threadID); // on disposal

// to create a makeshift thread for any object:
var vertices = [ /* ... */ ];
var texture = new Image("myTexture.png");
var shape = new Shape(vertices, texture);
var group = new Group([ shape ], shader);
Threads.createEx(group, {
    update: function() { return true; },
    render: function() { this.draw(); }
});

// to block until a thread finishes:
// (it is safe to call .join() in a thread updater--that thread will block
// but other threads will continue happily humming along!)
Threads.join(threadID);
Title: Re: minisphere 1.1b2 (stable: 1.0.10)
Post by: Fat Cerberus on May 02, 2015, 10:24:07 am
And here's a polyfill for another minisphere 1.1 feature: the RNG object.  Of particular note: RNG.fromNormal(). 8)

[gist=779ddef439c8b93f393b][/gist]
Title: Re: minisphere 1.1b3 (stable: 1.0.10)
Post by: Fat Cerberus on May 03, 2015, 01:37:53 am
minisphere 1.1b3 is now up and available to download. Tons of new features in this one (and a couple more bug fixes too)!  The RNG object is one I'm particularly proud of.  It's based on MT19937 (AKA Mersenne Twister), has a bunch of different generation methods, and allows manual seeding unlike JS's Math.random().  See the included API documentation for full details on it, it's pretty awesome.

I also took yet another leaf from TurboSphere's book and started on a JS-based "minisphere Runtime", previewed in this beta.  1.1b3 includes minithreads and minidelegate, the functionality of both of which have been invaluable during development of Spectacles and which I imagine will be useful in many, many games.  I'm thinking the final 1.1 release will also include a version of Scenario as part of the runtime as well.  These scripts can be used like this:

Code: (javascript) [Select]
RequireSystemScript('mini/threads.js');
RequireSystemScript('mini/delegate.js');


Also, minisphere is now officially BSD-licensed.  I figured it was about time I added a proper license file to the repo. :P


@DaVince:
1.1b3 fixes the BindKey() bug you reported on GitHub. :D
Title: Re: minisphere 1.1b3 (stable: 1.0.10)
Post by: Radnen on May 03, 2015, 03:50:03 pm
There are three things I want to see fixed sometime before I can even use this engine.

1. Fix the windowstyle bug for windows that have edge widths/heights less than 16 pixels. (See attached).
2. Make GetPixel() much faster so all code I have written that incorporates pixel perfect collision can make my game playable. It's too slow right now.
3. Fix a weird blit bug. (See attached). Things are offset by half the width of an image, but in other engines that's not the case. Like the hp bar, it's offset wrong. What do you get from image.width/2?

Edit: I will say that I'm impressed. You've really made it work on many games. :) Sometimes it's down to a single feature that a game doesn't run, so good you've got most features implemented.
Title: Re: minisphere 1.1b3 (stable: 1.0.10)
Post by: Fat Cerberus on May 03, 2015, 05:15:24 pm
That rotation bug was a known issue at one point... Then I forgot about it. :P  I think Allegro's treatment of the origin during rotation blits is off or something.  Either that or more likely, I interpreted it wrong.

The dashed windowstyle... Funny story about that.  I didn't really do much testing of Blockman in Sphere 1.5, I was too determined to get it running in minisphere.  As a result, I thought that was what it was SUPPOSED to look like.  If I had to guess, I'm thinking Allegro is padding out small bitmaps to be 16x16 under DirectX, as the windowstyle oddities doesn't happen with an OpenGL build of Allegro.

And the GetPixel() optimization I keep forgetting to get around to.  Thanks for the reminder!

And yes, I was VERY dedicated to making it compatible early in development.  I spent literally half a day trying to figure out a bug that caused Aquatis to crash on the opening credits.  It turned out to be a stupid refcounting error. :P
Title: Re: minisphere 1.1b3 (stable: 1.0.10)
Post by: Fat Cerberus on May 04, 2015, 01:52:42 am
I fixed the Image:rotateBlit() issue.  It was user error, as I suspected: Allegro treats the X/Y for a rotated blit as the center of rotation, whereas Sphere apparently aligns it to the top-left.  It should have been fixed a long time ago, but I forgot all about it once I started working on the map engine.

Moving on... I added pixel caching for Surface:getPixel().  It definitely helped, but it's still ungodly slow.  It might help if you stopped creating a surface anew with each collision check.  Allegro stores all bitmaps on the GPU, so each time you call Image:createSurface(), a new texture has to be uploaded.  Not only that, but it's putting a lot (and I mean A LOT) of stress on the garbage collector: I added some logging for images, and... well, let the console output speak for itself:

Code: (text) [Select]
Last 4 lines after entering test map:
[image 4731] Cache miss!
[image 4731] Creating new pixel cache
[image 4731] get_image_pixel() using pixel cache
[image 4731] Dropped pixel cache

Last 4 lines after ONE attack:
[image 5868] get_image_pixel() using pixel cache
[image 5885] get_image_pixel() using pixel cache
[image 5868] get_image_pixel() using pixel cache
[image 5885] get_image_pixel() using pixel cache


So during processing of a single attack, the game created (and possibly destroyed, depending on how aggressive Duktape's GC decided to be) over 1,000 bitmaps.  The thing died right away, so my guess is a few attacks got coalesced due to lag, but still: 1,000+ bitmaps!  That's insane.  That's literally insane.  No amount of pixel caching I can do is going to help that.  I honestly have no idea why--or HOW--it works so well in Sphere.

HOWEVER, full disclosure--I don't think this is the only issue.  What exactly goes on under the hood when you swing the sword?  I'm consistently able to drop the framerate to 30-40 fps on my i3 just by slashing at thin air, but I get no log entries that a pixel cache was created during this.

The last thing I have to fix is that windowstyle glitch.  I'm not sure how to deal with that yet.  I may end up switching to an OpenGL build of Allegro.  The only thing that worries me is that OpenGL tends to be slow on Windows, certainly slower than D3D.  But it would fix the windowstyle issue, so....

Edit: Yep, same issue with the slashes--simply slashing at thin air creates and destroys nearly 100 bitmaps.  Sorry to say, this game is VERY poorly optimized. :(  If it were me, what I would do is call .createSurface() once for each relevant sprite frame and then cache those somewhere.  This way you can just pull them from the cache and do .getPixel() on them with impunity.  As long as the surface contents never change (which they shouldn't), you'll never get a pixel cache miss and Allegro won't have to upload hundreds of textures for every attack.
Title: Re: minisphere 1.1b3 (stable: 1.0.10)
Post by: Radnen on May 04, 2015, 02:53:04 am

Edit: Yep, same issue with the slashes--simply slashing at thin air creates and destroys nearly 100 bitmaps.  Sorry to say, this game is VERY poorly optimized.


I'm a bit unnerved, really. I honestly don't know what to say. But I'll get to that in a sec, first an explanation. I'll be gentle, don't worry.

So I have no idea how thin air slashing does that. I mean it's just COMMAND_ANIMATE frames. I mean, I did nothing outside the bounds of the map engine for the action system (except the pixel collision detector). I use Pythagorean distance first to check if there are frames to collide with before this whole hundred-sprite-creation thing begins to happen

Are you saying this, on thin air creates and destroys 100 bitmaps?
Code: (javascript) [Select]

        // search and return a list of enemies in a 16 pixel radius at the front of the character. This is Attack()
        var list = this.getNearest(16, player.xv*16, player.yv*16);
        while (i--) {
            enemy = list[i];
            if (this.check(this.input, enemy.name)) { // call the check method
                //...
            }
        }


Thin air slashing ought to put the sword well away from the nearest enemy sprite. And how on Earth could it create a hundred to a thousand frames? Look at the code and think about it. I call the attack script only once on frame change.

Code: (javascript) [Select]

            // search for and add functions to certain frames.
            dir = "attack" + dirs[i];
            for (var n = 0; n < 7; ++n) {
                if (n == 4)
                    this.actionmanager.addFrameCall(dir, n, Cut);    // frame 4: cut
                else
                    this.actionmanager.addFrameCall(dir, n, Attack); // all others: check collision on Attack (see above)
            }

// later on in actionmanager:
// LOOK at it. It only does actions on frame switch: (because a sprite frame can last up to 8 screen frames)
var frame = GetPersonFrame(this.attached);
if (this.frame != frame) {
var index = GetPersonDirection(this.attached) + ":" + frame;
if (this.framecalls[index]) this.framecalls[index]();
this.frame = frame;
}


Technically my code only has to create bitmaps 2 times per frame (your source and the opponent source). I mean, I'm sure it'll create no more than 10 bitmaps max. I think your engine needs some work here. I mean, there are at least 100 to 1000 pixels to zip through on a dead collision check call, if it occurs. Are you telling me that each time I get a pixel your engine must create and destroy a bitmap? You have power over that my friend, not me. Especially if blank calls to COMMAND_ANIMATE does it. Well, technically running in your engine I can try to mitigate that, but at this point I'm not sure what's possible. I can cache the 2 bitmaps, but then I'm saving these 10 creations each time, the enemy constantly changes, mind you. So I could cache the payer. But if it's really each time a pixel is returned...I'm not sure. I don't want to have to lug around huge caches of enemies and players, or heck, maintain a cache pool if I can help it.

The "unoptimized section"
Code: (javascript) [Select]

        var ss = GetPersonSpriteset(player);
        var dir = ss.directions[GetDirectionNum(ss, GetPersonDirection(player))];
        var frame = dir.frames[GetPersonFrame(player)];
        var image = ss.images[frame.index].createSurface();
       
        // person B:
        ss = GetPersonSpriteset(who);
        dir = ss.directions[GetDirectionNum(ss, GetPersonDirection(who))];
        frame = dir.frames[GetPersonFrame(who)];
        var image2 = ss.images[frame.index].createSurface();


Above surfaces are created but only 2. Only 2 new bitmaps are created per call of check. Per the frames it's called on. Which is precisely frames 1 through 6 except 4. So 5 frames, which is no more than 10 bitmaps being created and thrown away. I check in SSFML, and yep, only 10 bitmaps. There is no lag. I usually don't try to optimize if there's nothing too crazy to optimize.  The code I use here is also industry standard (using pythos first, then making sure you only check an intersection of bounding boxes. In fact I doubt more than 256 pixels are ever checked!!). It's really industry leading. It's quite good, my friend of friends. Assuming of course bitmap creation is relatively cheap. If it's not, I can fix this, and you're right about it being unoptimized, but I'd say only slightly so. Because it's either this or cache pooling, and do I really want to do that if this works well enough? I mean each frame the enemy and player changes, and then each direction, for each enemy. There are so many states each bitmap can be in at any given time I'd rather reconstruct than cache, especially if reconstructing like it is in SSFML and Sphere1.5 is as dang fast as it is.

Because...

I only optimize code I've profiled for, looking for bottlenecks. ;) It's really what any sensible programmer would do. You saying the game is not very optimized kinda stings since I strive for that, but only in areas where I needed it. I say your engine is what's unoptimized here buddy since my SSFML handles this even better than Sphere's. The difference being I've used SW bitmaps. Now, I've moved on to HW sprites and noticed compositing was really fast but simple things like updating and creating new bitmaps as well as getting pixels was really slow. So I'm moving back to SW for good even if it means it's "slower".

Edit:
I profiled getPixel in SSFML and between 6000 to 2000 pixels are checked. Which is fine. Worst case scenario is 48*48*10 which is 23000ish pixels. Swinging empty air does nothing. No pixels, no bitmaps, nada.
Title: Re: minisphere 1.1b3 (stable: 1.0.10)
Post by: Fat Cerberus on May 04, 2015, 08:51:23 am
Sorry, I didn't mean to offend you--the game itself really is awesome, and the thousand bitmaps shocked me, too.  I'm only able to go by what the few logging calls I've added, but I honestly have no idea why it's creating so many.  Hell, I can just walk around aimlessly and get about 5-10 "created image via clone" log entries per second.  It makes no sense, because this doesn't happen in Specs or any other Sphere game I've tested.  Just yours.  Blockman is doing something differently but I don't know what yet.  I don't doubt the issue is on my end, though.

What's weird is that it's NOT generating multiple images for the actual pixel check--a successful attack generates a long string of pixelcache hits on the same two images.  It's just that it seems to be creating a lot of them in-between each check.  Weird.

Totally agreed on not making premature optimizations, don't take that the wrong way.  I myself advocate the "don't do it" school of optimization--it's the reason minisphere's codebase is still so clean. :D  I admit I jumped to conclusions when I said the game was unoptimized, but as I said, this is literally the ONLY game I've tested that causes so many images to be generated in such a short timeframe.  Out of probably a hundred games, both big and small.  "Engine bug" was naturally the last thing I would have considered.

So yeah, I thank you for clarifying the under the hood operation--I will now run the MSVC profiler to figure out where all the images are being generated.
Title: Re: minisphere 1.1b3 (stable: 1.0.10)
Post by: Fat Cerberus on May 04, 2015, 12:18:43 pm
So, I found part of the problem.  Take a look at this line in clone_spriteset():
Code: (c) [Select]
if ((clone->images[i] = clone_image(spriteset->images[i])) == NULL) goto on_error;


Spriteset bitmaps are never written to (only replaced wholesale), so the image cloning was completely unnecessary.  I changed the offending clone_image() call to ref_image() and that sped things up a ton.  My bad! :-[

Now, if you're wondering why I'm cloning spritesets at all, well, this calls for some background: As you know, I strive for compatibility, and the semantics of Get/SetPersonSpriteset() in Sphere are... odd, to say the least.  If you call GetPersonSpriteset() and change stuff around (replacing images, directions, etc.), the changes aren't reflected immediately--you have to call SetPersonSpriteset() to commit the changes.  And then, if you make further changes after that, you have to set it again.  This is ridiculous and completely counterintuitive, but I have to emulate it to remain compatible in odd corner cases (Aquatis was a wonderful source of these), and doing so means cloning the spriteset every time someone calls either function.  And since you use GetPersonSpriteset() a lot... yeah.

One thing I still notice, even with this change, is that I get a lot of this when just walking around aimlessly:
Code: (text) [Select]
[image 5416] Read 16 x 16 image from file
[image 5417] Read 16 x 16 image from file
[image 5418] Read 16 x 16 image from file
[image 5419] Read 16 x 16 image from file
[image 5420] Read 16 x 16 image from file
[image 5421] Read 16 x 16 image from file
[image 5422] Read 16 x 16 image from file
[image 5423] Read 16 x 16 image from file
[image 5424] Read 16 x 16 image from file
[image 5425] Read 16 x 16 image from file
[image 5426] Read 16 x 16 image from file
[image 5427] Read 16 x 16 image from file


To clarify, those are direct reads of an embedded image from an open file (such as happens when loading a tileset or spriteset).  Not sure what's up with that, probably something else stupid I'm doing somewhere.

There is a silver lining, of course... I guess it's a sign of a mature project when you to get to the stage of having to deal with low-level issues like this. :)
Title: Re: minisphere 1.1b3 (stable: 1.0.10)
Post by: Flying Jester on May 04, 2015, 01:09:01 pm

Moving on... I added pixel caching for Surface:getPixel().  It definitely helped, but it's still ungodly slow.  It might help if you stopped creating a surface anew with each collision check.  Allegro stores all bitmaps on the GPU, so each time you call Image:createSurface()


This is why Surfaces are meant to be totally in software. They are supposed to be relatively quick to create, destroy, and manipulate, much faster than Images.
Title: Re: minisphere 1.1b3 (stable: 1.0.10)
Post by: Fat Cerberus on May 04, 2015, 01:18:30 pm


Moving on... I added pixel caching for Surface:getPixel().  It definitely helped, but it's still ungodly slow.  It might help if you stopped creating a surface anew with each collision check.  Allegro stores all bitmaps on the GPU, so each time you call Image:createSurface()


This is why Surfaces are meant to be totally in software. They are supposed to be relatively quick to create, destroy, and manipulate, much faster than Images.


It would be literally trivial for me to have Allegro create surface bitmaps in memory... but wouldn't blitting Images to Surfaces then be SLOW since you're crossing the hardware boundary?  I guess I should do some tests, thanks for the tip.

Edit: Yep, that did the trick!  No pixel caching needed! ;D (scratch that, the pixel caching is still needed even with s/w surfaces--apparently al_get_pixel() is slow no matter what.

@Radnen: In case it wasn't clear, your getPixel() woes are fixed in the latest build.  Now I just have to tackle that windowstyle issue...
Title: Re: minisphere 1.1b3 (stable: 1.0.10)
Post by: Flying Jester on May 04, 2015, 02:48:50 pm

It would be literally trivial for me to have Allegro create surface bitmaps in memory... but wouldn't blitting Images to Surfaces then be SLOW since you're crossing the hardware boundary?  I guess I should do some tests, thanks for the tip.


Drawing Surfaces is OK to be slow. It's notably slower drawing Surfaces than Images in Sphere 1.5.
That is one of the tradeoffs of Surfaces, and why they are different than Images. Surfaces should be quick to read and write and cheaper to manage, Images should be fast to draw. Other performance aspects are generally irrelevant.
Title: Re: minisphere 1.1b3 (stable: 1.0.10)
Post by: Fat Cerberus on May 04, 2015, 04:56:26 pm
@Flying Jester
I knew there had to be a fundamental difference between the two (surface and image) beyond the superficial "you can write to one but not the other".  I just couldn't figure out what until you specifically pointed it out.  Luckily the way Allegro is set up, it was trivial to have it allocate surfaces in system memory instead of on the GPU.  Internally it's still an image_t structure, just that the underlying bitmap is created with different flags.

Honestly I'm pretty dumb for not figuring this out sooner.  There's a reason there are methods to convert between the two types!
Title: Re: minisphere 1.1b3 (stable: 1.0.10)
Post by: Radnen on May 04, 2015, 08:11:07 pm

but as I said, this is literally the ONLY game I've tested that causes so many images to be generated in such a short timeframe.  Out of probably a hundred games, both big and small.


I'm glad it was a good test case then. :) My game is really the only mature Sphere ARPG. Other games even Aquatis and T&E use separate views for their battles (take place in turn based arenas) so spriteset related action based things might not happen in the same manner. And I doubt they use pixel perfect collision to the degree that I do.

BTW Lord English I do understand in Sphere and am willing to accept that drawing surfaces and converting to surfaces is expected to be slow and is on the users end to mitigate those calls with their own crafty solutions. But creating a few surfaces here and there ought not to be slow. The next test in Blockman is entering a cave and seeing if the fullscreen darkness "shader" plummets the fps. I'm not the only game to do this since I used SDHawk's (blast from the past) advice on it.

Edit: BTW, running around in Blockman does create dust plumes under his feet, these can be what triggers those surface creations when walking around. Though I don't know why surfaces are created... It does load a new spriteset each time rather than reuse one. It's a stupid Sphere thing since CreatePerson forces you to load a spriteset per entity. I cache spritesets in SSFML, but it has the issue that perhaps each entity can share modifications made against them. Though I haven't seen that in the games I've tested, most games usually copy or reload from file a new spriteset via LoadSpriteset() whenever they modify a users sprite.
Title: Re: minisphere 1.1b3 (stable: 1.0.10)
Post by: Fat Cerberus on May 04, 2015, 09:26:28 pm
They aren't surfaces created, they are images, read as raw RGBA data from an open file.  Three things do this: load_tileset(), load_spriteset() and load_windowstyle().  So yeah, thanks for the tip on the dust plumes, I hadn't noticed those.  :)  There is a way you can avoid recreating new entities every time, just use SetPersonVisible and reuse them... But I agree it's pretty dumb that it doesn't let you pass in a Spriteset object to CreatePerson().  I think I'll go implement that now.
Title: Re: minisphere 1.1b3 (stable: 1.0.10)
Post by: Radnen on May 05, 2015, 12:16:21 am

There is a way you can avoid recreating new entities every time, just use SetPersonVisible and reuse them...


Neat idea, but you can have more than 2 dust plumes on screen so I'd have to craftily cycle anywhere between 2 to 4 them and that's a kind of complexity I didn't want to do. It seems that there is one pervasive issue with Sphere's API. Without client-side pooling and caching most calls to the Sphere API copy and clone data with reckless abandon. It's either you trust the underlying engine does a good job handling that for you, or you write lots of boilerplate whenever you need some simple effect.


... it's pretty dumb that it doesn't let you pass in a Spriteset object to CreatePerson().  I think I'll go implement that now.


That's a good step in the right direction. Oftentimes and in most other game engines many duplicates of objects like bushes, grass, footsteps in sand or snow, coins, etc., reuse the same asset to reduce the memory footprint. So, being able to directly tell Sphere that we intend to reuse the same spriteset in any number of entities can really aid in speeding things up. Creating and destroying entities is usually a cheap thing to do, I mean it really only encodes a few properties (mainly positional, nothing a matrix can't solve, really).

Also, imagine drawing to screen 30 entities with the same sprite atlas. You'll have sped drawing up considerably since you aren't changing the source texture each time. Games like DaVince's Sir Boingers can be sped up with this technique.

Now, I'm not sure here but Sphere has to cache Spritesets like my SSFML does. I can't imagine loading the same spriteset 300 times in a giant room with lots of coins in it. But who knows? It might do that? I'm not sure, I'll have to profile a demo.
Title: Re: minisphere 1.1b3 (stable: 1.0.10)
Post by: Radnen on May 05, 2015, 12:50:00 am
Results of my test:

Loading a map with 50 non-trivial entities. Same spriteset!
1. SSFML (30ms, then 5ms repeatedly)
2. Sphere1.5 (6ms, then 4ms repeatedly)
3. Minisphere (300ms then, 190ms repeatedly)

With 1 entity:
1. SSFML (30ms, then 5ms repeatedly)
2. Sphere1.5 (6ms, then 4ms repeatedly)
3. Minisphere (10ms, then 5ms repeatedly)

I say the hit with our engines is any sprite batching, vertex mapping, or texture appropriations due to HW acceleration methods.

Edit
With 2 different spritesets the numbers went up linearly initially. Only adding different spritests did Sphere 1.5 take longer to load. Repeating the same spriteset added no time.
Title: Re: minisphere 1.1b3 (stable: 1.0.10)
Post by: Fat Cerberus on May 05, 2015, 01:04:58 am


... it's pretty dumb that it doesn't let you pass in a Spriteset object to CreatePerson().  I think I'll go implement that now.


That's a good step in the right direction. Oftentimes and in most other game engines many duplicates of objects like bushes, grass, footsteps in sand or snow, coins, etc., reuse the same asset to reduce the memory footprint. So, being able to directly tell Sphere that we intend to reuse the same spriteset in any number of entities can really aid in speeding things up. Creating and destroying entities is usually a cheap thing to do, I mean it really only encodes a few properties (mainly positional, nothing a matrix can't solve, really).


Done.  It a rather simple change, actually.  Just some additional bookkeeping, and even that wasn't really an issue either as spritesets are already refcounted internally.

While we're on the subject of profiling and performance, you know how Blockman takes an coon's age to load fonts under minisphere on startup?  I fixed it.  Turns out the reason it was so painfully slow was because I was locking and unlocking each individual subimage (fonts are atlased) when loading characters.  That's 256 lock/unlock cycles, meaning data has to be uploaded to GPU every single time.  Now I lock the entire atlas ONCE, load all the glyphs, and then unlock it.  That's only one GPU upload.  The "Loading Fonts" text goes by so fast now you can barely even see it. ;D

Now those slow map loads have me wondering...

Edit: Okay, map loads are much faster now too, after an edit to the tileset loader.  It was the same issue as with fonts, the issue was just less pronounced as most tilesets don't have 200+ tiles in them and even then, you're only loading one at a time.  I should make a similar change the spriteset loading code, but it will be more difficult as spritesets aren't atlased.  At the time, it didn't seem worth it to atlas them since you're only usually drawing one image from a spriteset at a time.  But I might just do it anyway now to speed up map loading...
Title: Re: minisphere 1.1b3 (stable: 1.0.10)
Post by: Fat Cerberus on May 05, 2015, 08:25:08 am
I just refactored minisphere's image locking.  It's safe to nest locks, any lock requests past the first just reuse the existing lock, which is refcounted.  Allegro would just outright refuse to lock the bitmap a second time, which is how my atlased loaders ended up so slow.  Now I can lock the atlas at the beginning of the loader, and the individual read_subimage() calls will just reuse the existing lock.

This is awesome, as now I can easily implement spriteset atlasing. :D

Edit: Not only have I implemented sprite atlasing, but I also went ahead and added a spriteset caching feature.  Radnen, could you run your map load tests again on the latest source?  I'd be curious to see the results.
Title: Re: minisphere 1.1b3 (stable: 1.0.10)
Post by: Fat Cerberus on May 05, 2015, 08:08:06 pm
@Radnen
Figured out your windowstyle issue.  It's an Allegro limitation, and unfortunately, it's not a bug.

From Allegro 5.1.9 ogl_bitmap.c:
Code: (c,7,8) [Select]
   /* This used to be an iOS/Android only workaround - but
    * Intel is making GPUs with the same chips now. Very
    * small textures can have garbage pixels and FBOs don't
    * work with them on some of these chips. This is a
    * workaround.
    */
   if (true_w < 16) true_w = 16;
   if (true_h < 16) true_h = 16;


Unfortunately this means windowstyles can't be tiled in hardware--I have to do it manually and take the performance hit.

Alternatively, I could comment out those two lines and see what happens.  I have an Intel GPU laptop (Core i3), so I'll know if anything's amiss.
Title: Re: minisphere 1.1b3 (stable: 1.0.10)
Post by: Radnen on May 05, 2015, 08:18:52 pm

Unfortunately this means windowstyles can't be tiled in hardware--I have to do it manually and take the performance hit.


Tile it in hardware anyways. If you witness sizes less than 16, switch to manual. But anyways 16 was the sphere windowstyle default. I usually use that size and larger, but sometimes things slip through.
Title: Re: minisphere 1.1b4 (stable: 1.0.10)
Post by: Fat Cerberus on May 05, 2015, 11:49:37 pm
v1.1b4 is out, boasting more bug fixes, massively improved performance, OpenGL rendering on Windows and one of my famous subtle-but-awesome changes:

RequireSystemScript() now searches <game_dir>/scripts/lib for scripts first, before the system scripts directory.
Title: Re: minisphere 1.1b4 (stable: 1.0.10)
Post by: Radnen on May 06, 2015, 01:17:00 am
So I did a stress test in Sphere 1.5, SSFML 0.9.0, and minisphere 1.1b4

This is running at 26fps with 500 entities in SSFML
https://www.dropbox.com/s/cnaylep3xytgqdl/stress-ssfml.png?dl=0

This is running at 21fps with 250 entities in Minisphere
https://www.dropbox.com/s/8p9u3kzejhu8nvv/stress.png?dl=0

This is running at 17fps with 250 entities in Sphere 1.5 in SphereGL
https://www.dropbox.com/s/4nflt2g7cg4mkgc/stress-sphere15.png?dl=0

All fps's are +/- 5.

And well... Sphere 1.5 couldn't do 50 on Standard32. This test was on Intel Core i5 3.3@3.6ghz, with an Nvidia GTX970, 8GB 1066@800Mhz ram. I think it's the JS logic that's bottlenecking the two good engines, and the graphics plus the logic for Sphere 1.5. In SphereGL I got by no different than minisphere, which is telling. You are not taking advantage of the hardware on the computer, it tells me you're still mostly drawing in immediate mode which is what kinda plagued SphereGL and Sphere in general. I know this since I know your JS engine is faster in most cases to Sphere 1.5.

My max target for the space game is 300 enemies on screen. But this rest is more of a curiosity. In the game you never see more than 20 anyways, unless there's a giant space battle programmed into it. But I think the stress test is still neat. No engine crashed BTW.

Ok, so... minisphere 1.1b4 loaded blockman much faster! The combat was also nice and smooth. :) But it did crash after playing for a while when I swung my sword at an enemy. So there's something there, maybe a get pixel call out of bounds?
Title: Re: minisphere 1.1b4 (stable: 1.0.10)
Post by: Fat Cerberus on May 06, 2015, 01:35:32 am
I could speed up the windowstyle loading too if I atlased them, but then hardware tiling doesn't work.  The main loading bottleneck was fonts and spritesets (and maps), which have been speed up a ton not only due to atlased loading in 1.1b4, but also spriteset pooling.

But yeah, most of the drawing is done in immediate mode because for some strange reason, Allegro doesn't always honor transformations when using batched mode.  This is needed because I use a transform to scale up the window for 320x240 games.  I'd rather take the performance hit than end up with dodgy rendering.

I assume you noticed your windowstyles render properly now?  I bit the bullet and compiled Allegro with the 16x16 texture clamping commented out.  So far I haven't had any issues, so... *crosses fingers*

As for those crashes, I've seen a few of them myself, but I haven't tracked down the cause yet.  Hence why minisphere 1.1 is still in beta. ;)
Title: Re: minisphere 1.1b4 (stable: 1.0.10)
Post by: Radnen on May 06, 2015, 02:50:10 am
You might want to stay in immediate mode for now. I'm taking a small risk batching everything. I'm meaning to create a small batching API to make it explicit. But I have found through my testing that even implicit batching is decent enough, and really only helps when you have much of the same sprite, like the above where the enemies and particles and bullets were the same 3 textures.

I guess if you stay in the map engine, implicit batching really is not needed since sprite atlasing covers that. The implicit batching I do to speed up the space game only really works when I program knowing that this behavior exists and that I can draw images in a custom engine rather than use spritesets in the map engine. There is a slight speedup drawing names on their own layer rather than in an A/B pattern. But it's minimal.
Title: Re: minisphere 1.1b4 (stable: 1.0.10)
Post by: Fat Cerberus on May 06, 2015, 03:12:25 am
So it didn't make it into 1.1b4, but I just added a console and BGM manager to the mini runtime, both originally coded for Specs.

The console is output-only (for now), but optionally lets you log output to a file, and because it uses minithreads, lets you pull up the console at any time during gameplay.  This little tool has been invaluable in debugging Specs glitches, and is generic enough that I thought it'd make a good addition to the runtime. :)

The BGM manager right now is a bit of a fixer-upper, but the intent is to turn it into a simple stack-based music switcher.  You push to change the song, and pop to return to the previous one (perhaps resuming where that one left off).  I might also add a "layer 0" as well for field music, which can be changed in place but only plays when there are no other tracks on the BGM stack.

That puts the minisphere runtime at 6 components:
- Core (provides delegates)
- Link
- minithreads
- miniscenes (Scenario)
- miniconsole
- miniBGM

Quite a nice set of tools it's shaping up to be, methinks.
Title: Re: minisphere 1.1b4 (stable: 1.0.10)
Post by: Fat Cerberus on May 06, 2015, 03:45:46 pm
So I got bored and implemented basic command support in miniconsole.  It's a very easy-to-parse command format--e.g. battle rsb2 for a test battle in Spectacles.  So now anyone that wants to implement a basic console in their game will have one ready to go!

Usage:
Code: (javascript) [Select]
RequireSystemScript('mini/Console.js');

mini.initialize({
    logFile: "consoleLog.txt",
    consoleLines: 10,
    consoleBuffer: 1000
});

mini.Console.register('bear', {
    'munch': DestroyPerson
});


Then the player could pull up the console and type in:
bear munch lauren

And the map person named "lauren" would get eaten by the bear and disappear.  How it is parsed is, the first token is the object name (bear) as passed to mini.Console.register(), the second token is the command name (munch), and the remaining tokens are passed as string parameters to the relevant method.  The actual call is also wrapped in a try/catch to try to prevent game crashes due to bad console input.

Fixed tt tag ~radnen
Title: Re: minisphere 1.1b4 (stable: 1.0.10)
Post by: Fat Cerberus on May 08, 2015, 03:01:56 am
minisphere 1.1 is going to be delayed by a bit, it seems.  I apparently broke something when adapting Scenario and now it locks up sometimes during scene execution.  I haven't been able to track down the cause.  It's odd, the game seems to completely lock up, but closing the minisphere window works, which means FlipScreen is being called.

I suspect a bad interaction between the Scenario threader and minithreads is causing a deadlock.  I didn't even think that was possible with cooperative threading (making a distinction between a deadlock and an infinite loop, before anyone calls me on it), but...
Title: Re: minisphere 1.1b4 (stable: 1.0.10)
Post by: Fat Cerberus on May 09, 2015, 01:53:20 am
So I think I'm going to end up releasing minisphere 1.1 without the runtime.

As it stands right now, all runtime components rely heavily on threads.  And as I've just found out, minithreads has a rather massive design flaw: Because join is emulated with recursion, joins don't always do what you expect: Assuming two simultaneous joins, satisfying the first one is--by definition of recursion--necessarily contingent on the second one returning.  Even if the first join would otherwise be satisfied first, in practice it can't be because one further down the rabbit hole is still blocked.  Sometimes--aw, who am I kidding, way too often--this can even cause a deadlock.

For whatever the reason, as long as I've been using this threader in Specs, this never bit me before now.  I guess Specs alone just didn't tax the system enough to flush it out--it wasn't until I tried to reimplement Scenario in terms of joins that it showed up, and struck HARD as Scenario absolutely requires reliable concurrency.  What sucks is that it took me all day to figure it out, after many exhausting hours of misguided hacking trying to fix what couldn't be fixed.

Curious how I finally figured it out?  I added two GetSeconds() calls to Scenario's tween scenelet, one in start and another in finish.  And lo and behold, one particular tween that was supposed to take half a second... didn't finish up until 12 seconds later.  I puzzled for a bit, and then it dawned on me what was going on... and I felt like a complete idiot. :-[
Title: Re: minisphere 1.1b4 (stable: 1.0.10)
Post by: Radnen on May 09, 2015, 02:04:52 am

Curious how I finally figured it out?  I added two GetSeconds() calls to Scenario's tween scenelet, one in start and another in finish.  And lo and behold, one particular tween that was supposed to take half a second... didn't finish up until 12 seconds later.  I puzzled for a bit, and then it dawned on me what was going on... and I felt like a complete idiot. :-[


That's not being an idiot, sometimes sanity checks once and a while help. Now, how are you going to solve it, that's the question. Deadlocks usually arrive because of a lack of concurrency handling, every threaded system has mutex locks, semaphores even busy-wait loops, you just need to pick solution(s) that work.

But I don't think this is a deadlock. It might be a race condition since after 12 seconds it finished rather than .5... a deadlock is most usually... dead! A race condition means things are vying for cycles and other loops/threads get pushed back. In fact I'm sure that 12 seconds is variable. Is it 13 next time, immidiate another, 5 seconds?
Title: Re: minisphere 1.1b4 (stable: 1.0.10)
Post by: Fat Cerberus on May 09, 2015, 02:24:51 am
Me being an idiot wasn't referring to the sanity check, it was that, knowing how recursion works, this eventuality never even crossed my mind.

But the 12-second tween indeed wasn't a deadlock.  However, deadlocks most certainly can (and DO) happen with this setup.

Here's the implementation of join():
Code: (javascript) [Select]
mini.Threads.join = function(threadIDs)
{
threadIDs = threadIDs instanceof Array ? threadIDs : [ threadIDs ];

if (!this.isInitialized)
Abort("mini.Threads.join(): must call mini.initialize() first", -1);
var isFinished = false;
while (!isFinished) {
this.doFrame();
isFinished = true;
for (var i = 0; i < threadIDs.length; ++i) {
isFinished = isFinished && !this.isRunning(threadIDs[i]);
}
}
};


Note that it loops until the requested thread(s) finish.  This works well enough--if there is only one outstanding join.  Trouble is, Scenario is often waiting on many things at once.  At this point, the recursive system breaks down.

Let's simplify things a bit and say we have four running threads, call them A, B, C and D.
* A joins B, which prevents A from being updated until B terminates.  So far so good.
* .join() is running its own loop, so B, C, and D keep updating.
* At some point (A is still waiting on B to finish), C joins D.
* B terminates.  A's join now can't be satisfied because, by definition of recursion, it is itself blocked by C's join.  D may not terminate until some way down the line, which means it's unintentionally holding up the works.

And yes, this system can give rise to a deadlock very easily.  In the simplest case, this will do it:
* A joins B.
* B joins A.

In practice, such cycles will be (and were!) much more roundabout and therefore difficult to debug.

Note that, due to the way the system is designed, the effects of this may not be immediately obvious as other threads will continue to update (just like preemptive threading!).  However, if what A is trying to accomplish is necessary for the game to continue (hence why B wanted to wait on it in the first place), this will put the game in an unrecoverable state.
Title: Re: minisphere 1.1b4 (stable: 1.0.10)
Post by: Fat Cerberus on May 09, 2015, 11:40:15 am
Speaking of sanity checks, I added Assert yesterday.  The API signature:
Code: (javascript) [Select]
Assert(condition, errorText[, stack_offset]);


I made the error text mandatory instead of optional to encourage meaningful asserts.  If an assert fails (the condition is false), the specified error text will be displayed in a modal popup allowing the user to choose whether to continue or terminate.  As with Abort(), the optional stack offset is used to blame the failure on a function earlier in the call stack.

Hm... now that I think about it, Assert should probably behave differently in the console vs. release engines.  In a release build (engine.exe, engine64.exe), a failed Assert should terminate the engine unconditionally (without giving you the yes/no options), while the Console build (console.exe, console64.exe) would log the assert to stderr and give you the option to try to continue.

As for that threader issue, what I think I'm going to do is modify the semantics of join.  Rather than block the caller directly, it will store the joined tids in the calling thread's "join list", and that thread will not get any further update() calls until all the joined threads have terminated.  This will require a few architectural changes to Specs, but so be it.  I'll definitely miss the blocking .join() though.  It almost perfectly emulates Pthreads join, but that "almost" turned out to be the killer.
Title: Re: minisphere 1.1b4 (stable: 1.0.10)
Post by: Radnen on May 09, 2015, 05:26:21 pm
I have found out that in our engines, SSFML and minisphere, this causes a considerable speedup to getPixel:

Code: (javascript) [Select]

function SurfaceWrap(image, hold) {
this.surface = image.createSurface();
this.width = image.width;
this.height = image.height;

var pixels = [];

this.update = function() {
var w = this.width, h = this.height;
for (var x = 0; x < w; ++x) {
pixels[x] = [];
for (var y = 0; y < h; ++y) {
pixels[x][y] = this.surface.getPixel(x, y);
}
}
}

if (!hold) this.update();

this.getPixel = function(x, y) {
return pixels[Math.floor(x)][Math.floor(y)];
}

this.rotate = function(r, s) {
this.surface.rotate(r, s);
this.update();
}

//... other methods...
}


I think it has to do with the domain bounds, between script engine and native execution because even with native side pixel caching, engine based caching is faster to retrieve from. The only downside to the above is, notice the rotate, I have to call the update method each time to get a fresh list. But my code only rotated the image once. The speedup isn't huge, though. Testing on 500 static images, the engine used to halt entirely in our engines. With this addition there is no visible halting, everything is just slow, like 10-17fps slow, but consistent which says this is much faster. Though in practice it may be slower, especially if you modify the surface a lot with rotations and other methods (I only wrapped the rotate method).

In Sphere1.5 the above errors with an out of memory error, after about 40 entities.
Title: Re: minisphere 1.1b4 (stable: 1.0.10)
Post by: Fat Cerberus on May 09, 2015, 07:21:02 pm
This makes sense.  When calling getPixel() directly, even though the engine is caching it, you still have the overhead of a function call per pixel.  In Duktape, a JS->C call is identical to a JS-JS call except for the last step (call into native instead of bytecode)--in fact it may even be faster because some of the call setup can be skipped (arguments object creation, etc.).  But the fact remains that calling a function in JS is a lot more expensive than in native.  Caching the pixels locally avoids that, as then the JS engine just has to retrieve values from an array, which is trivial.

By the way, the out of memory errors in 1.5... You will see those a lot.  It only allocates 5MB to the JS context!
Title: Re: minisphere 1.1b4 (stable: 1.0.10)
Post by: Fat Cerberus on May 11, 2015, 03:35:30 pm
I just added a new API: DoAsync().  This queues a script to be executed on the next FlipScreen.  It's like what setTimeout(fn, 0) does in a browser.  I added this mostly because of minipact, since according to the Promises/A+ spec, promise resolution must be asynchronous, outside the usual program flow.

I'm sure this has other uses, though... :)
Title: Re: minisphere 1.1b4 (stable: 1.0.10)
Post by: Fat Cerberus on May 12, 2015, 03:48:02 pm
So using promises for minithreads turned out to be inadequate.  They work well enough for Scenario which creates a bunch of one-off threads and runs them in parallel, only occasionally stopping to resync timelines.  But for more complex threading, like waiting on a menu thread during a battle, things get awkward because the battle thread still updates in the meantime.

What I'm going to do is change the semantics of join().  It will still return a promise, but it will ALSO exclude the calling thread from further update() calls until the threads being waited on finish.  This way join() would still work mostly like it does in Pthreads without holding up the works with recursive event loops, and the promise would take care of post-join logic (such as queuing a move, etc.)

Edit: On second thought, I'm not even going to do that.  Blocking join works fine for 90% of use cases, it's only interlacing joins that cause issues.  I'll just make a note not to do that in the documentation. :)
Title: Re: minisphere 1.1b4 (stable: 1.0.10)
Post by: Fat Cerberus on May 13, 2015, 02:00:13 am
@Radnen
I figured out the Blockman crashes.  The getPixel calls were fine, none were out of range--though I added bounds checking anyway, since Sphere also checks it (an out of range getPixel() will throw in 1.5).  The crash was here:

Code: (c) [Select]
void
update_persons(void)
{
bool is_sort_needed = false;

int i;

for (i = 0; i < s_num_persons; ++i) {
if (s_persons[i]->leader != NULL)
continue;  // skip followers for now
update_person(s_persons[i]);
is_sort_needed |= has_person_moved(s_persons[i]); // <-- CRASH HERE
}
if (is_sort_needed) sort_persons();
}


As part of update processing for persons, they can end up being deleted (if a command generator script destroys one, say), which is exactly what happened here.  Once I reproduced the crash, I checked the variables.  i = 3 and s_num_persons = 3, which meant one was deleted, but it was still trying to access the person structure, causing a crash.  Should be an easy fix.
Title: Re: minisphere 1.1b5 (stable: 1.0.10)
Post by: Fat Cerberus on May 13, 2015, 02:45:03 am
1.1b5 is out.  A couple bug fixes in this one, as well as two new APIs: Async() (required for promise compliance, see changelog) and Assert(), very useful for debugging.  This will be the last beta ahead of 1.1, now I want to concentrate on finishing up the API documentation before release, and maybe write that config tool...
Title: Re: minisphere 1.1b5 (stable: 1.0.10)
Post by: Fat Cerberus on May 16, 2015, 02:43:24 am
I decided to go in a different direction with the customizable key mappings.  I added a new API, SetPlayerKey(), to change the key mappings at runtime; any such changes are then saved out to disk on shutdown.  What I'm thinking is, I will add commands to miniconsole so that any game using the minisphere Runtime can have its keymap changed by the player through the console.  miniconfig as a separate thing will still exist, of course, but its scope will be enlarged: Allowing key mappings to be changed for any game minisphere has previously run or otherwise knows about.

This will allow me to work on miniconfig at my own pace, rather than having to rush it to make the 1.1 release.  I can release it separately when it's ready. :-)
Title: Re: minisphere 1.1b5 (stable: 1.0.10)
Post by: Radnen on May 16, 2015, 02:58:04 am
I like the idea. I've always wanted many config related options to be game specific. Hence why I never used the config.exe tool to set up key mappings, that and having the ability to set them and save them in game is always nice.
Title: Re: minisphere 1.1b5 (stable: 1.0.10)
Post by: Fat Cerberus on May 16, 2015, 11:52:23 am
See the screenshot below.  Any games using miniconsole will get these commands by default. :D
Title: Re: minisphere 1.1
Post by: Fat Cerberus on May 17, 2015, 02:07:26 am
It's been a long ride, but minisphere 1.1 is finally ready for primetime! ;D  So um... go download it!  See the changelog in the OP for all changes since 1.0.x, but the highlights are:


The download is a lot bigger this time (as in over 7 MB) not only because of the new Console engine but also because it includes a new Spectacles battle engine demo to show off the power of miniRT. :D
Title: Re: minisphere 1.1
Post by: Fat Cerberus on May 18, 2015, 12:11:51 pm
@Radnen
So I finally got around to implementing gradient circles, and was therefore able to test out the cave lighting effect in Blockman.  Results: Success!  Not only does the effect look great, but it runs at full speed.  See screenshot below. ;D
Title: Re: minisphere 1.1
Post by: Fat Cerberus on May 19, 2015, 05:22:50 pm
So, um... I got animation support working. Which means the KR intro now works! ;D

Damn is libmng bloated though.  It added about 200KB to the engine executable size.
Title: Re: minisphere 1.1
Post by: DaVince on May 19, 2015, 06:01:59 pm
Amazing and awesome. KR is a buggy mess that likes to not work properly on some versions of Sphere, so it might be an interesting test case for minisphere. :P
Title: Re: minisphere 1.1
Post by: Flying Jester on May 19, 2015, 08:17:46 pm
Did you add your own mng support, or was it in Allegro?


Damn is libmng bloated though.  It added about 200KB to the engine executable size.

;D
Title: Re: minisphere 1.1
Post by: Fat Cerberus on May 19, 2015, 08:19:27 pm
No, I had to do it myself using libmng and loading the frames into locked surfaces.  You'd think Allegro would have it, but nope.  Luckily I was able to use the Sphere source as a reference again, to lessen the learning curve a bit.

Edit: It's funny, I wasn't originally going to do it, I was just going to have some stub APIs in place to allow games using the functions to get by, but once the stubs were in place, I said ah, why not.  I'm used to the hassles of compiling dependencies on Windows by now, so what's another library? :P

Edit 2: I got the 64-bit Windows build of minisphere back down to under 1.7MB by rebuilding everything optimizing for size instead of speed. :D  I don't appear to have taken a visible performance hit by doing so, but time will tell...
Title: Re: minisphere 1.1.1
Post by: Fat Cerberus on May 20, 2015, 12:33:50 am
minisphere 1.1.1 is up.  This adds the animation support and finally fixes GradientCircle, which I've been too lazy to fix up until now.  Also, the windowstyle rendering should be fixed on all platforms as I added a workaround (i.e. tile in software) for images < 16x16.

I've decided to go to an Allegro-style "tick-tock" versioning cycle for minisphere: Even-numbered versions (e.g. 1.0) are stable and only receive bugfixes, while odd versions (e.g. 1.1) are considered WIP and can get new features (or have them removed) at any time.  I like this setup because it allows more public testing to be done on new features before they get committed to a genuine Stable release.
Title: Re: minisphere 1.1.1
Post by: Fat Cerberus on May 21, 2015, 11:42:49 pm
Just added shader support today.  :D

Code: (javascript) [Select]
var shader = new ShaderProgram('pixel.glsl', 'vertex.glsl');  // this might be changed to take an object argument
var image = new Image('ScottSucks.png');
var shape = new Shape([
{ x: 10, y: 10 },
{ x: 110, y: 10 },
{ x: 110, y: 110 },
{ x: 10, y: 110 },
], image);
var group = new Group([ shape ], shader);


Initial tests are promising!
Title: Re: minisphere 1.1.2
Post by: Fat Cerberus on May 23, 2015, 02:03:23 am
minisphere 1.1.2 out now, now with GLSL shaders!

You know, I'm beginning to think I'm releasing these faster than anyone can test them... :P  Good thing my own in-house tests are very thorough!
Title: Re: minisphere 1.1.3
Post by: Fat Cerberus on May 25, 2015, 02:23:14 am
1.1.3 has been posted.  Just a maintenance release, nothing groundbreaking this time. ;)

At this point it seems the only thing I'm really missing is a particle engine.  Sphere 1.6 had one from what I understand, and that's one thing that would definitely benefit from being implemented natively, I think.  It can be done in script, but I'd imagine it might be slow then.
Title: Re: minisphere 1.1.3
Post by: Fat Cerberus on May 25, 2015, 10:37:43 pm
Look what I got running! ;D

It was a pain because a lot of the map scripts included extended characters, which confused Duktape (it expects valid UTF-8 always), so I had to have the engine re-encode the embedded scripts on load.
Title: Re: minisphere 1.1.3
Post by: DaVince on May 26, 2015, 07:19:24 am
Amazing. Great work. I need to try this out right away. When the new release is out. Heh.
Title: Re: minisphere 1.1.3
Post by: Fat Cerberus on May 26, 2015, 01:00:40 pm

Amazing. Great work. I need to try this out right away. When the new release is out. Heh.


You got your wish--1.1.4 is posted, with a few other nifty fixes.

As for KR, it's a very interesting testcase indeed.  You can get it to run, but it requires a bunch of minor edits all over the place.  You'll find that there are a lot of lines like this:
Code: (javascript) [Select]
SetPersonLayer('turtletrainer', '3');


Which of course breaks in minisphere, and probably Sphere 1.6, because both support named layers.  If this were consistent, I could write it off as the game making misguided assumptions about the underlying engine... but then I see just as often:

Code: (javascript) [Select]
SetPersonLayer('roofus', 3);


A correct invocation.  And this kind of thing happens EVERYWHERE.  I'm beginning to think Keith had a weird sense of humor and did this kind of thing on purpose just because he could.
Title: Re: minisphere 1.1.4
Post by: DaVince on May 26, 2015, 02:16:15 pm
Considering how the code is full of jokes (like new retard()), that wouldn't surprise me. Then again, he also told me that he was shitty at coding. KR was the project that helped him learn, and he didn't fix old code. That really turns things into a mess.

And thanks a lot for the new release. ;D

Edit: yep, this game is pretty broken. After fixing a line of SetPersonLayer() and saving a PNG camouflaged as a BMP as an actual BMP, the game runs, but there are lots of problems. Can't enter most doors, can't talk to most people. I'm just impressed it's working this well besides those problems.
Title: Re: minisphere 1.1.4
Post by: Fat Cerberus on May 26, 2015, 03:04:57 pm
Did you notice the cleanup I did on the error screen?  The error location is now set off from the message, which should make it easier to see the location of an error at a glance. :)
Title: Re: minisphere 1.1.4
Post by: Fat Cerberus on May 27, 2015, 12:12:25 am

Edit: yep, this game is pretty broken. After fixing a line of SetPersonLayer() and saving a PNG camouflaged as a BMP as an actual BMP, the game runs, but there are lots of problems. Can't enter most doors, can't talk to most people. I'm just impressed it's working this well besides those problems.


Actually, on further testing... I think I broke something.  Person-person collision is screwy.  I definitely remember being blocked by those NPCs on my initial playtests, but now you can walk right through most of them...  which also means you can't talk to them because the engine does a collision check to find persons to talk to.

Edit: Ah ha!  I figured it out.  This is a long-standing engine bug that it took this horrendously buggy game to uncover.  I must have been mistaken about the collision working before, that was probably one of my Sphere 1.5 tests before I got minisphere to run the game.  Anyway, the bug is that minisphere doesn't normalize the spriteset base.  Sphere does, and it seems KR has several spritesets with degenerate bases where x1 > x2 and/or y1 > y2.  This causes the obstruction checks to fail.

It took me forever to figure this out because I was operating on the assumption this was a layering bug.  Not even close! :P

See, this is what's made minisphere what it is: no matter how broken my test game is, I work on the basis that, if it works properly in Sphere but not in minisphere, then the issue must be on my end (I was perhaps too hard on Blockman earlier, but regardless, this applied there as well).  I'm still very dedicated to compatibility. :)  Of course, there are two cases where I draw the line: I won't relax type checking for API arguments (you're calling into C code, the engine has every right to police it), and I won't emulate bugs.
Title: Re: minisphere 1.1.4
Post by: Radnen on May 27, 2015, 02:35:46 am
But if a bug means a game can be playable... It's a tricky situation. You want a clean version of Sphere with no bugs in it, but if you have to incorporate a bug to make a game work... it's not always straightforward isn't it? But then again I guess you could call those bugs "features". :P
Title: Re: minisphere 1.1.4
Post by: Fat Cerberus on May 27, 2015, 02:58:13 am
It can sometimes be a very fine line to walk, I won't deny that.  For example, the color-cycling trick in Sphere is based on what is technically a bug (namely: byte overflow in color components isn't checked for) and when it was brought to my attention I had to think long and hard whether to emulate it.  Ultimately I chose not to in the name of consistency (CreateColor can reasonably be considered "canon" and that clamps the values), which of course breaks compatibility.  However, it's a trivial enough thing to work around (you can do modulo arithmetic yourself), so I don't consider it a dealbreaker.

I guess this is the reason browsers have quirks modes...
Title: Re: minisphere 1.1.4
Post by: Fat Cerberus on May 27, 2015, 11:58:10 am
Hm, so it appears minisphere's handling of triggers is borked, which is why you can't enter buildings in KR.  Unfortunately this doesn't surprise me, that was one area I didn't do much in-house testing on.  My test map for Specs does include a trigger, but it's only one.  KR tends to cluster them together, and it seems the activation rectangle for individual triggers is so small for whatever reason that they often don't activate at all.

Edit: I did some runs through the debugger.  The issue is a weird race condition between KR's custom movement code (I have to give whoever programmed the movement system credit, they basically independently invented Lithonite) and an optimization in minisphere where I only check for trigger activation if the input person has moved since the last frame.  For whatever reason the person almost always reports as not moving.  I'm not 100% positive yet, but I THINK the root cause is that the game confuses the engine by constantly detaching and attaching input.
Title: Re: minisphere 1.1.4
Post by: Fat Cerberus on May 28, 2015, 01:37:39 am
Going to make a new post for this because I want it to be seen.  This was quite an interesting pursuit.

Got it!  I was right, KR detaching and reattaching input during custom movement throws it off.  Currently minisphere has an optimization (premature, as I now see) where it runs a trigger check if and only if two conditions are met: 1) There is an input person, and 2) that person has moved since the last frame.  In this case, when the engine runs the trigger check, no input is attached because the game reattaches it in a person script.  And that script runs on the frame AFTER all movement has ceased:

Code: (javascript) [Select]
QueuePersonCommand(thePC.name, COMMAND_MOVE_NORTH, false); // <-- is_immediate is false, suspends command execution until next frame
QueuePersonScript(thePC.name, "trigtest(); if (!Kbusy && !climbmode[0]) AttachInput(thePC.name); walking=false; ", false);


Note the trigtest(); in the reattachment script.  This would normally mitigate the issue, but the function is bugged.  Long story short, this game is very, VERY lucky to be playable at all, even in vanilla Sphere.  The ONLY reason it works in vanilla is because Sphere always walks the trigger list on every update, even if the input person is stationary.  It looks like I will have to do the same if I want to support this corner case.

So yes, this game is broken, but the fact that it works in the original engine makes it interesting: In many cases it's hanging on to its stability by a thread, so the smallest difference in engine behavior under the hood can break everything.  If ever there were a testcase to punish premature optimizations and incomplete reverse engineering, this is it. :P
Title: Re: minisphere 1.1.4
Post by: Radnen on May 28, 2015, 02:27:49 am
I'm  not sure trigger checks need this kind of optimization. I'd let it check as often as I want, after all it's hardly a bottleneck. The more I get away with in native, the better.

What I am interested in here is the reattachment of the player. When the input is not attached, triggers are not executable through the engine (I end up calling ExecuteTrigger() or somesuch). If the idea of the above is to trigger a trigger a frame before the input is attached I'm not sure how that goes. I guess it's the re-attachment of the player that suddenly executes the trigger because the check is indeed being made all the time, every frame. Once it sees the input is reattached, bam, it's triggered. I think that's kind of clever, really.
Title: Re: minisphere 1.1.4
Post by: Fat Cerberus on May 28, 2015, 02:34:39 am
Yeah, I'm not sure what the purpose of the trigtest() function is there, it's completely redundant as Sphere will run the trigger anyway post-attach.

And now that I think of it, I don't think it was an optimization after all: the same condition ("did the input person move?") covers zone processing as well.  And there it's 100% necessary, otherwise the zone script will keep firing even if the player stands still.
Title: Re: minisphere 1.1.4
Post by: DaVince on May 28, 2015, 06:22:27 am
Quote
So yes, this game is broken, but the fact that it works in the original engine makes it interesting: In many cases it's hanging on to its stability by a thread, so the smallest difference in engine behavior under the hood can break everything.

This actually explains perfectly why certain segments of the game started breaking randomly on later Sphere versions. For example, I remember the couch moving scene later on in the game sometimes working, sometimes not.

It'd probably be better to fix the game here rather than your code specifically, especially if it means performance regression for all other games. I mean, the game needed some fixes anyway.
Title: Re: minisphere 1.1.4
Post by: Fat Cerberus on May 28, 2015, 10:51:41 am

It'd probably be better to fix the game here rather than your code specifically, especially if it means performance regression for all other games. I mean, the game needed some fixes anyway.


In this particular case it's not a big deal, as you'd incur the performance hit, such as it is, of constant trigger checks anyway as long as the PC is moving (which is why I now suspect it wasn't an optimization at all, just an oversight).  The only speedup to be had from this is when the player is standing still, which truth be told, isn't very useful.

However, in the general case, I agree with the sentiment.  I'm not about to bend over backwards to get this game to run as-is, but if it should uncover what is clearly an engine bug, I do want to try to fix it. :)

Trivia: Technically my trigger implementation isn't fully compatible with Sphere's, as I found out last night when comparing the two engine sources.  Sphere tracks current trigger activation per-person, whereas minisphere only has a single "current trigger" variable.  What this means is that, in Sphere, if the PC steps on a trigger which causes input to be attached to another person, if the input is later returned to the original person, the trigger won't reactivate.  In minisphere it would because trigger activation is tracked globally.

Edit: Yeah, even with the trigger and zone fixes I just committed, there's no way this game is going to run in minisphere without some serious edits.  There's way too much of passing strings to APIs expecting numbers.  Regardless, it looks like I've fixed all of the actual bugs: The game looks like it would be playable if not for all the type errors.
Title: Re: minisphere 1.1.5
Post by: Fat Cerberus on May 28, 2015, 03:50:15 pm
Okay, v1.1.5 has been posted, and with it I've decided to declare minisphere 1.1 stable.  Most of the KR glitches are fixed by this release and outside of that my tests haven't uncovered any showstoppers, so I'm thinking it's time to retire v1.0.  The performance improvements in 1.1 alone are worth the upgrade.  I'll leave 1.0.11 in the downloads repo, but minisphere 1.0 should be considered no longer officially supported and won't receive any more fixes.

Go crazy!

(Note that I didn't bump the version to 1.2, as I was originally going to do for stable releases.  I decided it was an unnecessary formality.)
Title: Re: minisphere 1.1.5
Post by: mezzoEmrys on May 28, 2015, 05:50:52 pm
TechDemo_Lithonite2.0 works perfectly on engine.exe, but crashes engine64.exe after selecting to play it from startup. console64.exe informs me that it reaches "Initializing input" before the crash.

Edit: Just realized I can try to debug it with Visual Studio, which I actually forgot I had on here, because I've never really used it.
Anyway, it had this to say:
Unhandled exception at 0x000000013F997AB9 in console64.exe: 0xC0000005: Access violation reading location 0x0000000000000055.
Title: Re: minisphere 1.1.5
Post by: Fat Cerberus on May 28, 2015, 08:17:31 pm
It crashed at initializing input, interesting.  And this is the 64-bit engine... Hm.  I know that happens on XP, preventing the engine from being used there.  For some reason on some systems it likes to segfault when initializing the Allegro joystick routines, and there's nothing I can do because it's internal to Allegro.

There's no difference between engine and engine64 other than the latter is a native x64 binary and is theoretically faster as a result.  Functionality is the same, so you lose nothing. :)
Title: Re: minisphere 1.1.5
Post by: Fat Cerberus on May 29, 2015, 03:58:22 pm

TechDemo_Lithonite2.0 works perfectly on engine.exe, but crashes engine64.exe after selecting to play it from startup. console64.exe informs me that it reaches "Initializing input" before the crash.

Edit: Just realized I can try to debug it with Visual Studio, which I actually forgot I had on here, because I've never really used it.
Anyway, it had this to say:
Unhandled exception at 0x000000013F997AB9 in console64.exe: 0xC0000005: Access violation reading location 0x0000000000000055.


mezzo, could you redownload 1.1.5 and try engine64 again?  I just uploaded a hotfix that should fix the crash you're seeing.  In theory this should also allow the engine to run on XP...
Title: Re: minisphere 1.1.5
Post by: Flying Jester on May 29, 2015, 05:06:20 pm
...is a native x64 binary and is theoretically faster as a result...


Them's fightin' words! =P
Title: Re: minisphere 1.1.5
Post by: Fat Cerberus on May 29, 2015, 05:14:50 pm
Fighting words?  Why?
Title: Re: minisphere 1.1.5
Post by: Flying Jester on May 29, 2015, 05:52:04 pm
Because so many people disagree on it...
Title: Re: minisphere 1.1.5
Post by: mezzoEmrys on May 29, 2015, 05:54:02 pm

I just uploaded a hotfix that should fix the crash you're seeing.


Doesn't seem to work, the only difference seems to be that the error is occurring at 0x0[zeros]019 instead of 0x0[zeros]055, and as far as I know that could just be a memory shuffling thing.
Title: Re: minisphere 1.1.5
Post by: Fat Cerberus on May 29, 2015, 06:57:38 pm

Because so many people disagree on it...


Hence why I said "theoretically".  ;)
Title: Re: minisphere 1.1.5
Post by: Fat Cerberus on May 29, 2015, 07:04:11 pm


I just uploaded a hotfix that should fix the crash you're seeing.


Doesn't seem to work, the only difference seems to be that the error is occurring at 0x0[zeros]019 instead of 0x0[zeros]055, and as far as I know that could just be a memory shuffling thing.


Huh.  I know there was a bug I found in Allegro where if it fails to initialize XInput it will crash instead of falling back on DirectInput like it's supposed to.  Long story short, it initializes XInput and then doesn't check the return value from the init function.  I fixed it in my own copy of the Allegro source and recompiled, and it got past that step on my XP VM, so maybe I uploaded the wrong file...

Do you have the latest DX runtimes installed?  Contrary to what you'd think, even Win8.1 is missing parts of it out of the box.
Title: Re: minisphere 1.1.5
Post by: mezzoEmrys on May 29, 2015, 07:28:59 pm

Do you have the latest DX runtimes installed?


I think I do, I seem to have 11.1 based on what information I could find. I'm not sure what else I could do from there to ensure I have the latest I can for Win7.
Title: Re: minisphere 1.1.5
Post by: Fat Cerberus on May 29, 2015, 09:07:55 pm
Hm, there definitely shouldn't be any issues under Win7.  Oh well, you said the 32-bit engine (engine.exe) works fine, right?  I guess you'll have to use that.  Like I said, they are both compiled from the exact same code, so you won't lose any functionality.

Let me know of any other issues. :)
Title: Re: minisphere 1.1.5
Post by: Fat Cerberus on May 29, 2015, 11:25:09 pm
So it might not be too long before minisphere gets module support:
https://github.com/svaarala/duktape/issues/201

;D

ArrayBuffer support is also slated for the next release, which is awesome.
Title: Re: minisphere 1.1.5
Post by: Radnen on May 29, 2015, 11:34:05 pm
64bit version IS faster on my machine. :)
Title: Re: minisphere 1.1.5
Post by: Fat Cerberus on May 29, 2015, 11:47:07 pm

64bit version IS faster on my machine. :)


Good to know. :D
Title: Re: minisphere 1.1.6
Post by: Fat Cerberus on May 31, 2015, 01:30:20 am
So I was able to reproduce the crashes mezzoEmrys saw with the 64-bit engine.  Apparently Allegro doesn't appreciate being fully shut down and then reinitialized in the same session, which is what happens when ExecuteGame() is called.  At least for me, the crash doesn't always happen, only sometimes.  I will have to refactor the way ExecuteGame() works (I'll put it on the checklist for v1.2), but for now there shouldn't be any issues outside of startup games.  In general, actual games have no need to call ExecuteGame().

Anyway, minisphere 1.1.6 is up, and with the recent fixes it should now work in Windows XP. 8)  @Defiant, if you are still watching this topic, could you test it for me?
Title: Re: minisphere 1.1.6
Post by: Fat Cerberus on May 31, 2015, 03:09:29 am
I'm just a regular juggernaut with this project, aren't I?  I just implemented module support, slated for minisphere 1.2:
https://github.com/fatcerberus/minisphere/tree/module-support

I converted kh2Bar to a module to test it out and so far it's working as intended.  This is awesome!

At present minisphere loads modules from ~/modules (or failing that, ~#/modules i.e. relative to the engine "system" dir).  I was originally going to use ~/scripts/lib, but as I already used that as a preferred search path for RequireSystemScript, I figured a separate directory was the better option.  That'll make it easier if someone wants to code up a NuGet-style dependency manager, too.
Title: Re: minisphere 1.1.6
Post by: Fat Cerberus on May 31, 2015, 11:39:19 pm
So the more I play around with modules, the more I realize, they're not always a suitable replacement for old-fashioned RequireScript().  For one-off dependencies that are only used in maybe one or two places, e.g. kh2Bar, it makes sense to use a module, but for more pervasive dependencies (Link, miniRT...) I've found that a RequireScript() at the top of the main source file is much clearer than var whatever = require('whatever') (or worse, a bunch of inline require()s littered throughout the source which Pegasus seems to advocate).
Title: Re: minisphere 1.1.6
Post by: Flying Jester on June 01, 2015, 12:49:58 am

So the more I play around with modules, the more I realize, they're not always a suitable replacement for old-fashioned RequireScript().


This is particularly true with smaller libs. What we have right now is much more like a < script > tag, which for many things is more suitable. But we actually go a step further with RequireScript over EvaluateScript.
For something like my 1.5-style map engines, or (I'd suspect) your threader a module would be more suitable. But for my older MJ map engine, it would be silly.

That's kind of why I don't really care too much about module support. I don't really consider the namespacing to be too important, since it's pretty simple to do that yourself (and all modules I've ever seen already do), and we already handle a basic but quite effective form of dependency management.
Title: Re: minisphere 1.1.6
Post by: Fat Cerberus on June 01, 2015, 01:10:14 am
The thing I really love about RequireScript is that, besides providing built-in protection against multiple inclusion, it can also be used to enforce load order in complex libraries.  In miniRT, for instance, the very bottom of miniRT.js has this block:

Code: (javascript) [Select]
// now that everything is in order, we can pull in all the miniRT
// component scripts.
RequireSystemScript('mini/miniBGM.js');
RequireSystemScript('mini/miniConsole.js');
RequireSystemScript('mini/miniLink.js');
RequireSystemScript('mini/miniPact.js');
RequireSystemScript('mini/miniScenes.js');
RequireSystemScript('mini/miniThreads.js');


The components are listed in alphabetical order, however miniThreads actually has to be initialized first, because everything else in the runtime depends on it.  And it indeed does get initialized first because the individual component scripts also have a RequireScript for it, and therefore it gets its initializer registered with the runtime before any other component.  This kind of thing I found is very difficult to do with modules because of the focus on encapsulation.

Nonetheless, this was a trivial addition I could have done ages ago (it's built into Duktape, and even if not it could be polyfilled easily using RawFiles and eval), and implementing it now puts me in a very good position to start work on some form of Pegasus support.
Title: Re: minisphere 1.1.7
Post by: Fat Cerberus on June 01, 2015, 12:35:57 pm
@mezzoEmrys
Could you try the latest release (1.1.7)?  I'm pretty sure I fixed the crash on restart now.
Title: Re: minisphere 1.1.7
Post by: mezzoEmrys on June 01, 2015, 07:30:16 pm
The release download link seems to be not happening. I also have no idea how to build it myself, and I'm not sure I even have the compilers myself. I really need to get on that...
Title: Re: minisphere 1.1.7
Post by: Fat Cerberus on June 01, 2015, 07:40:37 pm
The download doesn't work?  Odd.

You only need VS 2013 to build it. All the dependencies including Allegro static libs are included in the repo.
Title: Re: minisphere 1.1.7
Post by: mezzoEmrys on June 01, 2015, 07:47:56 pm
Your link on the first post is just http://, no actual link body, at least not one that comes through to me.
Title: Re: minisphere 1.1.7
Post by: Fat Cerberus on June 01, 2015, 07:49:05 pm

Your link on the first post is just http://, no actual link body, at least not one that comes through to me.


Oops! :-[

https://docs.google.com/file/d/0BxPKLRqQOUSNX3pTNHU1Uk1tdm8/edit?usp=docslist_api
Title: Re: minisphere 1.1.7
Post by: Fat Cerberus on June 01, 2015, 08:06:29 pm
Okay, download link is fixed.

For future reference, the latest release is always available in the SphereDev download repository.  You know, in case I post a broken link again... :)
Title: Re: minisphere 1.1.7
Post by: mezzoEmrys on June 01, 2015, 08:42:54 pm
Huzzah! All works as expected. I tested both games this time, and everything seems to work as expected.
Title: Re: minisphere 1.1.8
Post by: Fat Cerberus on June 02, 2015, 12:59:07 am
minisphere 1.1.8 is up.  This fixes diagonal movement commands not working as well as the long-standing issue where animation for diagonally-moving persons was too fast due to double-queuing of COMMAND_ANIMATE.

Note that, as a result of the recent changes (the way COMMAND_ANIMATE is handled internally has changed), persons will now keep animating even if they run into something, a la early Pokemon.  Technically this is a bug, but it does have a certain charm to it, so I'm not actually sure I want to fix it. :)  What do you guys think?
Title: Re: minisphere 1.1.8
Post by: Radnen on June 02, 2015, 02:33:28 am

Technically this is a bug, but it does have a certain charm to it, so I'm not actually sure I want to fix it. :)  What do you guys think?


I say the animation is fine, I try to emulate the behavior in my games since I like it. I caution if it affects Lithonite or not.
Title: Re: minisphere 1.1.8
Post by: Fat Cerberus on June 02, 2015, 02:59:03 am
It doesn't seem to, looks like Lithonite still works normally.

Edit: Huh.  That's appropriate--look at my post count. :P  I almost don't want to ever post again now, it's just too perfect. ;D (1025 being 413 + 612 and the date of Cascade, the most epic thing to ever happen in Homestuck)
Title: Re: minisphere 1.1.8
Post by: DaVince on June 02, 2015, 07:32:57 am
Oh, I thought it was 1024, a nice power of 2, and you already posted again since then or something. :P

I haven't had the time to play with the new releases. Once vacation starts, you can bet your ass I'm making a game powered by minisphere though.

Also, do you happen to have documentation for any of the new functionality? Or should I just be checking the older thread posts for now?
Title: Re: minisphere 1.1.8
Post by: Fat Cerberus on June 02, 2015, 08:34:46 am
Check the documentation folder for minisphere-api.txt.  Most of the new stuff should be documented, it's the legacy functions I skimped out on (which you have the 1.5 doc for anyway).  I'm slowly filling those out too as I go along though.
Title: Re: minisphere 1.1.8
Post by: DaVince on June 02, 2015, 10:07:41 am
Okay, thanks a bunch. :)
Title: Re: minisphere 1.1.8
Post by: Fat Cerberus on June 03, 2015, 02:05:05 am
minisphere 1.2 development is officially underway. ;D  The latest commits add the following APIs for Zone management:



Three of these (AddZone, RemoveZone and SetZoneDimensions) look like they were planned for Sphere at some point and experimentally implemented, but were never exposed to script.  AddZone in particular has the entire function commented out in the source. ???  So seeing that, I figured "Why not?" and implemented them.

SetZoneScript was inspired by this thread:
http://forums.spheredev.org/index.php/topic,1194.0.html

And Get/SetZoneSteps were just low-hanging fruit. :P
Title: Re: minisphere 1.2b1 (stable: 1.1.8)
Post by: Fat Cerberus on June 03, 2015, 03:10:19 pm
The first minisphere 1.2 beta is available, go check it out!  This includes the new zone and trigger APIs mentioned above as well as CommonJS support. :)  Oh, and the API documentation has been fleshed out some more.
Title: Re: minisphere 1.2b1 (stable: 1.1.8)
Post by: Fat Cerberus on June 03, 2015, 11:28:09 pm
So out of curiosity, I ran the loop tests I discovered hidden in the Sphere-SFML test game source with all three current engines.  The results are attached as screenshots.

The cliffnotes version: Obviously SphereSFML won because Jurassic is awesome, however one thing shocked me: The huge divide between minisphere and Sphere 1.5.  I'll get to that in a minute, though.  Before I get started, my test rig is an AMD A10-6800K @ 4.1 GHz with 8GB RAM and running Windows 8.1 Pro x64.

As you can see in the screenshots, Sphere 1.5 averaged around 1300ms per test.  The only ones that were under 1000ms were the two that increment twice per iteration.  This pattern shows across all engines, though, so apparently the loop condition check is expensive?

SSFML 0.90: The best result by far was "For i++ in Step", which got 17.2ms(!).  Makes sense, that's the common case for a for loop, so Jurassic optimizes for it.  What's interesting, though, is that Jurassic falters when it hits the i += 1 tests.  If you're JITting, then i += 1 should compile to the same machine code as an increment, right?  But apparently it doesn't.  Weird.  Ignoring the outliers, SSFML averages ~50ms per test, and around 130ms if you include everything.  Very impressive.

Now we get to minisphere (1.2b1).  If this is any indication, Duktape is literally 3-4 times faster than SpiderMonkey 1.5.  I knew it was a highly optimized engine, but this still blew me away.  Here again, though, i += 1 is way slower than a postfix increment for no discernible reason.  It's only a 100ms difference though, so not really an outlier.  Average across all tests: ~385ms.

So the bottom line is, not only is minisphere leagues faster than Sphere 1.5, but it holds up well against a JITted engine as well.  And from what I hear, the Duktape dev is working on even more performance improvements, so that gap should narrow even further in the future.
Title: Re: minisphere 1.2b1 (stable: 1.1.8)
Post by: Radnen on June 04, 2015, 01:17:50 am
It's funny how an engine not necessarily built around speed still out-performs Sphere 1.5. I guess SpiderMonkey team really just focused on getting it to work, not even they focused on optimization.

Jurassic lacks static type JIT optimizations. If you are doing a for loop and it knows you are doing integer adds it should use increment. This was on the table of optimizations to be made but the author never got around to it. I thought the C# JIT would do optimizations for you if you can supply it the MSIL bytecode, but perhaps it comes down to what environment or context you give it and the author hadn't optimized for these cases.

That's raw speed for you though. I've since moved to a different test where I call into a function like CreateStringFromByteArray(), the performance varied greatly. When I ran that in a loop in SSFML it took 400ms to do what Sphere 1.5 did in 20ms. But I found out that there is a way to rewrite the way Jurassic notices the method bindings (rather than letting it guess the methods, you can tell it, which is much faster) and upon doing that to my code, I was able to get it down to 40ms. Sphere 1.6 did the test in 60ms so obviously things got even slower between bindings from 1.5 to 1.6. The for loop test was faster in 1.6, however, making the benchmark rather odd.

So I'd do a benchmark with raw JS (which is this) and JS->native calls like CreateColor() or CreateStringFromByteArray() and just see what they bring to the table. Because, Sphere code will make use of many Sphere API calls in loops and they all ought to run fast or otherwise I'd just do -*everything* in JS and only expose file and input I/O as well as a basic drawing API and have the map engine and everything else live in script to remove the native call overhead. I think for Jurassic this might be the fastest since everything's IL anyways. But it's just the optimizations that it'll lack.
Title: Re: minisphere 1.2b1 (stable: 1.1.8)
Post by: Fat Cerberus on June 04, 2015, 01:27:52 am
Yeah, obviously I knew this benchmark was useless for real-world performance testing, I was just interested in raw performance of the JS engines.  I'll definitely try adding some API calls to the mix at some point though.

I do want to do a JS map engine at some point though, perhaps for minisphere 2.0.  I'll keep the built-in default one in the engine, but I want to try my hand at writing a more OO map system, which JS is far better suited to than C.  Things like SetDefaultPersonScript might be more difficult with objects, though...
Title: Re: minisphere 1.2b1 (stable: 1.1.8)
Post by: Flying Jester on June 04, 2015, 03:37:07 am

So I'd do a benchmark with raw JS (which is this) and JS->native calls like CreateColor() or CreateStringFromByteArray() and just see what they bring to the table. Because, Sphere code will make use of many Sphere API calls in loops and they all ought to run fast or otherwise I'd just do -*everything* in JS and only expose file and input I/O as well as a basic drawing API and have the map engine and everything else live in script to remove the native call overhead. I think for Jurassic this might be the fastest since everything's IL anyways. But it's just the optimizations that it'll lack.


A huge advantage of newer SpiderMonkey is that it is quite capable of this, since it's the stance Mozilla takes with the web now. JavaScript for everything.


Yeah, obviously I knew this benchmark was useless for real-world performance testing, I was just interested in raw performance of the JS engines.  I'll definitely try adding some API calls to the mix at some point though.

I do want to do a JS map engine at some point though, perhaps for minisphere 2.0.  I'll keep the built-in default one in the engine, but I want to try my hand at writing a more OO map system, which JS is far better suited to than C.  Things like SetDefaultPersonScript might be more difficult with objects, though...


You could always check out how I did it. My map engine has fairly complete API coverage (mostly missing FollowPerson stuff), even if it isn't totally accurate, and it is fully object oriented.
Title: Re: minisphere 1.2b1 (stable: 1.1.8)
Post by: Fat Cerberus on June 04, 2015, 08:14:39 am
Yeah, I'll probably take a more in-depth look at the Turbo engine at some point.  I would want an object-oriented API though as well though, not just under-the-hood.  For the most part the 1.5 interface is perfectly serviceable, but there are places where it's... awkward.  Things like Get/SetPersonX/Y and whatnot.

I do have to caution you that the person spooling you do pre-MapEngine() isn't 100% compatible with Sphere's behavior and is likely to catch you off guard at some point.  I know it did for me on multiple occasions during development.  Sphere--and minisphere--actually create a concrete person object (not just metadata) when you call CreatePerson(), and non-transient persons survive a MapEngine cycle.  This latter behavior is where the spooling will break down, I think.
Title: Re: minisphere 1.2b1 (stable: 1.1.8)
Post by: Fat Cerberus on June 04, 2015, 05:05:08 pm
Word of advice: When stubbing things out  during development, particularly lesser-used functions, make sure the stub can't be mistaken for correct functionality. :P This is what I did with font.clone(), and it literally JUST got fixed today.  The stub I wrote for it simply incref'd the original font and returned that.  Since fonts are rarely cloned and the behavior is fairly close to what it's supposed to do, I ended up forgetting about it.  The only, ONLY reason it got fixed was because I was writing the documentation for the Font object and happened to notice it while reviewing the code.

Other bugs fixed today as a result of the review: font.setCharacterImage() not updating the font metrics, font.wordWrapString() not handling whitespace properly and causing crashes under rare circumstances (namely, small wrap widths), and better handling of long stretches of characters with no whitespace.
Title: Re: minisphere 1.2b1 (stable: 1.1.8)
Post by: Fat Cerberus on June 05, 2015, 01:33:52 am
New minisphere 1.2 feature: the Sphere object.  This is an alias for the JS global object and is similar to window in a browser, allowing you to create global variables from strict mode code as well as serving as an optional namespace for built-in APIs if you're so inclined:

Code: (javascript) [Select]
Sphere.isThisAwesome = true;

var key = Sphere.GetKey();

Sphere.FlipScreen();

// etc.
Title: Re: minisphere 1.2
Post by: Fat Cerberus on June 06, 2015, 01:51:10 am
minisphere 1.2 is up.  I've decided to quit with the betas from here on out and just release.  minisphere has been on such a rapid release cycle anyway and I've been very good about fixing bugs, so there's really nothing to gain from pre-releases for this project.

Anyway, 1.2's flagship feature is CommonJS support.  This means require(); now works in minisphere if anyone is so inclined to use it. :)  I've also bumped the reported API version to 2.0.  It's not Pegasus, but between Galileo, CommonJS modules and the object constructors introduced in 1.1, I feel the engine identifying as lowly Sphere 1.5 is a bit disingenuous at this point. :P
Title: Re: minisphere 1.2
Post by: Fat Cerberus on June 06, 2015, 10:23:53 pm
I just discovered a neat trick for determining the correct facing command for a corresponding movement:

Code: (javascript) [Select]
var faceCmd = moveCmd - COMMAND_MOVE_NORTH + COMMAND_FACE_NORTH;


Note that this only works in minisphere.  Sphere is missing the diagonal movement commands, so the values don't line up. :(


Title: Re: minisphere 1.2
Post by: FBnil on June 07, 2015, 08:50:37 am
Congrats on the new release. I took it for a spin. Impressive.
Using Linux with wine. I had Full screen issues, due to the keyboard not binding to the screen, had to alt+tab out and in again.
Very much interested in seeing zones working. (saw a post in march that is was enabled... but 1.2 does not have it?)

About the speed with S1.5. It was build with JS1.4 (or was it JS1.5?). I succesfully tested JS1.6, 1.7 and 1.8rc1 through the years (before that fateful day my laptop blew up) and JS wise, they were faster, and also allowed new JS functions, but did nothing to the FPS of games (unless you were not using the mapengine, but most did), while using up much more RAM. A the moment, the API of JS changed so much (see the releasenotes from JS31) that lots of interface calls have to be rewritten. As Spidermonkey gets up to par with V8, I expect more from it now. But am discouraged, as I am not even getting zlib compiled without errors. (switched to MSVC++2010, God knows where my bought DVD is... never got to use it). Any way, the easiest way was ripping JS from old Firefox distro's. Some required also a NS4 dll, next to JS32.dll and some of those builds were slower.

https://developer.mozilla.org/en-US/docs/Mozilla/Projects/SpiderMonkey/Releases/31

For now, I think spending time on testcases is much more fruitful. I can not even get zlib to compile at the moment...  (zlib\src\contrib\masmx86\inffas32.asm(649): error A2070: invalid instruction operands)... so that is... bad... of course, I got a new MSVC, and do not remember what I required to set up to get it all working again. If somebody can point me to a wiki that works...
Title: Re: minisphere 1.2
Post by: Fat Cerberus on June 07, 2015, 10:07:16 am
Zones definitely work.  The Lithonite tech demo is included with the engine and should be fully functional.  Literally nearly the entirety of the 1.5 API is implemented, the only thing that generally stops a game from working is the stricter argument checking (Boolean arguments aren't automatically coerced, etc).

Check minisphere-api.txt in the documentation folder for full documentation, including all the new stuff. ;D

Not sure what all that was about SpiderMonkey and zlib though.  I'm guessing you're trying to build Sphere 1.x?  Wrong thread for that. :P
Title: Re: minisphere 1.2
Post by: FBnil on June 07, 2015, 10:44:38 am
Somehow... I rechecked the version: 1.2 does contain 2 binaries: 64 bits and 32 bits. Can not run it on VM Windows XP SP3 (64bits is not executable, as expected. and the 32 bits abends with "minisphere was unable to create a display window"). On wine, the 32 bit runs. But zones do not work. Packaged binaries are from Jun  6 01:35

I also detected a problem with layers that move automatically: When next to a map border, the map does not move, but the sprite does, instead of staying in the center of the screen, while the map moves, it is the sprite that moves while reaching a border. In those cases, the layers moves away from the cursor keys, as if the map was still moving. (need a example game for that?)
Title: Re: minisphere 1.2
Post by: DaVince on June 07, 2015, 10:44:51 am
I have a suggestion: would you consider a "global install" version of Sphere? One where you'd put all the games in an App Data folder on Windows or in $HOME/.local/sphere (or $HOME/.sphere) on Linux, and the actual executable in Program Files and /usr/bin? That would be useful for a few reasons:

A) It allows Sphere to become available in package managers, which in turn would attract new users
B) You can just run the command "minisphere" from anywhere that way
C) You separate the binary from the data that can be changed (like the games folder), which means no write access problems

Thinking about it, this is mostly beneficial for Linux. Though point C benefits both Windows and Linux, and probably OS X too.

Quote
Using Linux with wine.

That begs the question... why? The native Linux version functions perfectly fine. :P
Also, good to see you back!
Title: Re: minisphere 1.2
Post by: Fat Cerberus on June 07, 2015, 10:47:16 am
I have no idea why zones aren't working for you, nil.  Did you try the Lithonite tech demo I included with the engine?  That is based heavily on zones obviously, and works fine on my end.

@DaVince: This is a good idea, I'll consider it for the next release. :)
Title: Re: minisphere 1.2
Post by: Fat Cerberus on June 07, 2015, 10:53:32 am
Oh, and yes, I'd like an example game for the layer thing because I have no idea what you're talking about :P.  I assume the zone thing you're doing is some edge case where my behavior differs from Sphere's as well, so I'd like to see a testcase for that too.  I just tested a few games with zones myself and they work fine, so your use case must be different.

Oh, and yes, it won't run in an XP VM, I tried.  I think it can't initialize OpenGL due to lack of hardware acceleration, so Allegro display creation fails.
Title: Re: minisphere 1.2
Post by: FBnil on June 07, 2015, 11:36:54 am

That begs the question... why? The native Linux version functions perfectly fine. :P

Because I was testing minisphere 1.2 (and the zip contains only a windows build?). I am a bit lost though, lots of reading/catching up to do in 1 year. And all interesting posts, so I can not skip along...

Thanks for the welcome back! Good to see you guys again! Hope you are all doing well.

I attached a fog demo. Just run up, and you see the fox moving too fast, then, behave normally once the sprite is centered.

Will try to get it on a windows 7 machine to test it there. For now, no luck getting zones running. Just in case, can you point me again to the binaries... just in case (I grabbed them from the first post, that was updated... but...)
Title: Re: minisphere 1.2
Post by: FBnil on June 07, 2015, 11:38:42 am

Did you try the Lithonite tech demo I included with the engine?

Of course, and also one I had lying around. All I can think of is that I grabbed the wrong zip.
Title: Re: minisphere 1.2
Post by: Fat Cerberus on June 07, 2015, 12:10:04 pm
minisphere 1.2 download link:
https://drive.google.com/drive/folders/0Bw-4UFVty4u1fjJ2RHNUTFNzMVp1WEFVOHlJaFk4STJPd1RMVmFLeGtDQWQ5SVlSSXhEWFk

I've made a few changes since the release in the Git repo, but none that affect the map engine.  So yeah, this is baffling.  I see what you mean about the layer thing though.  I copied the parallax logic pretty much as-is from Sphere 1.x (in my own coding style of course :) ), so I must have screwed something up there.  I'll look into it and fix it and post minisphere 1.2.1.
Title: Re: minisphere 1.2
Post by: Fat Cerberus on June 07, 2015, 12:52:29 pm
Okay, I diagnosed the parallax bug.  Sphere does camera correction before parallax, whereas minisphere currently does it after.  This messes up the offsets when parallax is involved.  I should have caught this in testing, but as not many Sphere games in the wild use parallax and my own tests involved a repeating map (so there are no edges), it didn't show up.

It should be easy enough to fix.  Thanks for the report!
Title: Re: minisphere 1.2
Post by: FBnil on June 07, 2015, 01:47:54 pm
Not only with parallax enabled, if we take sully chronicles, the palmtree leafs on the upperlayer go everywhere too.
Another bug is an offset with the collision box. See image.

I also attach TechDemo_Lithonite2.2 (it has a mirror demo to the topright of the screen, the rest is the same. It has a newer version of Lithonite, 2.2 with a few bugfixes)
Title: Re: minisphere 1.2
Post by: Fat Cerberus on June 07, 2015, 01:58:44 pm
Ah, so it's a regression in GetCurrentZone(), not zone handling specifically.  That used to work (Lithonite was one of my big compatibility targets during development, just so you know :D), but apparently now it doesn't.  Thanks, that screenshot was a huge help!  On my end it ended up reporting it as Zone 769 half the time, which of course broke the mirror demo with an "invalid layer index" error.  I'll look into it.

Do you happen to know where I can get Sully Chronicles?  I don't see it in the Spherical repository.  I'm assuming it has to do with the parallax too though, maybe not parallax specifically but autoscrolling, which is part of the same calculation.
Title: Re: minisphere 1.2
Post by: DaVince on June 07, 2015, 02:06:35 pm
I should still have Sully Chronicles on my computer. Should. Gimme a minute.

Quote
Because I was testing minisphere 1.2 (and the zip contains only a windows build?). I am a bit lost though, lots of reading/catching up to do in 1 year. And all interesting posts, so I can not skip along...

Ah, okay. Basically, you just get the code from Git and compile it yourself - the dependencies are very minimal (only need the Allegro dev libraries and libmng).
Title: Re: minisphere 1.2
Post by: DaVince on June 07, 2015, 02:29:28 pm
Sully is now available in the incomplete/demo games folder: https://drive.google.com/open?id=0Bw-4UFVty4u1UGhwY2dKMFdySlE&authuser=0
Note that this game is VERY incomplete.

I took the liberty to test it out a bit too, though in an older version (on Linux, haven't updated/compiled it in a while). The layers were all screwy, and the mod music isn't playing. Gonna recompile and see how it is with the latest.
(Edit: yep, same deal with latest)
Title: Re: minisphere 1.2
Post by: Fat Cerberus on June 07, 2015, 02:46:33 pm
Yep, I was right, it was the parallax.  For SC, for whatever reason the parallax flag is set on the Foreground layer, so minisphere treats it as a repeating layer, while the map itself is non-repeating.  Since the parallax code didn't do preemptive camera correction like Sphere does (I thought it would be enough to only correct the final offset), this caused parallax layers to scroll out of sync with the rest of the map.  If these had been repeating maps, everything would have been fine.

Both the GetCurrentZone() bug and this issue are now be fixed as of this commit:
https://github.com/fatcerberus/minisphere/commit/3a9498d6daeea5264ecceed17026714071ec2a79

Thanks for the bug reports guys, as always.  :) minisphere 1.2.1 will be up shortly, with both of these fixes included.
Title: Re: minisphere 1.2
Post by: DaVince on June 07, 2015, 02:49:36 pm
It's great to see a game that didn't work correctly only ten minutes ago work correctly now. :P Hope you've noticed the mod music not playing? Because it's not under Linux, for some reason.
Title: Re: minisphere 1.2
Post by: Fat Cerberus on June 07, 2015, 03:05:41 pm
minisphere 1.2.1 is up.  It fixes the bugs discussed above as well as a bug in font.wordWrapString() causing the last word in the text to be lost if it ends on a wrap boundary.
Title: Re: minisphere 1.2.1
Post by: Fat Cerberus on June 07, 2015, 03:24:23 pm
Hm, you're right about the mod music not playing.  I suspect another Allegro bug, I will look into it later after work.
Title: Re: minisphere 1.2
Post by: FBnil on June 07, 2015, 04:40:22 pm
Nice work, thanks!

Quote

Will try to get the git and compile it myself...

Forgot to mention: make sure you get the "unstable" 5.1 version of Allegro. ;)
Title: Re: minisphere 1.2.2
Post by: Fat Cerberus on June 08, 2015, 02:21:15 am
minisphere 1.2.2 is up, which fixes the BGM in Sully Chronicals.  I never bothered to implement map BGM, so that's why it broke.  Nothing to do with the MOD music at all, those work just fine. :)
Title: Re: minisphere 1.2.2
Post by: DaVince on June 08, 2015, 08:09:37 am
Nice work, thanks!

Quote

Will try to get the git and compile it myself...

Forgot to mention: make sure you get the "unstable" 5.1 version of Allegro. ;)
Title: Re: minisphere 1.2.2
Post by: DaVince on June 08, 2015, 09:38:22 am
So I decided to try to make my ancient Pokémon GoldenSky/SilverSea project compatible with minisphere. This was basically my first big project, and where I gained learned the bulk of my scripting experience. And you can tell - the code is a mess all over. :P

Along the way, I noticed a few interesting things:
I declared var Font, var WindowStyle, var Surface. Needless to say, this worked in the original Sphere but minisphere complained because the objects are in use. I got this error, which baffled me until I finally figured out what was going on:
Code: [Select]
JS Error: TypeError: invalid base value


After fixing those (and also converting the used MIDIs to Ogg files), I was pleasantly surprised to see it RequireScript() through my dozen other script without complaining whatsoever. Awesome.

However, now I've stumbled across another difference between Sphere and minisphere: GetKey() is not aware of KEY_CTRL. Either that, or GetKey() doesn't return key codes at all. This means you cannot get past the menus. I filed a bug report. :)
Title: Re: minisphere 1.2.2
Post by: Fat Cerberus on June 08, 2015, 10:11:25 am
Yeah, that "invalid base value" error from Duktape is the bane of my existence.  What it means, specifically, is that you're trying to access a property on a variable which is not object-coercible (only null or undefined, in practice). It could at least tell you the name of the variable you're trying to access (SpiderMonkey does), but instead all you get is a vague "invalid base value".  Nothing I can do, unfortunately.  I'll have to file an issue on the Duktape repo about it.
Title: Re: minisphere 1.2.2
Post by: Fat Cerberus on June 08, 2015, 11:30:21 am

I declared var Font, var WindowStyle, var Surface. Needless to say, this worked in the original Sphere but minisphere complained because the objects are in use.


I have no idea how that ever worked.  By definition when you do that you're overwriting (or at least shadowing) the prototype on the Font constructor, so everything should break.  Yet somehow in Sphere that works normally... very odd.
Title: Re: minisphere 1.2.2
Post by: DaVince on June 08, 2015, 11:58:04 am
Original Sphere never exposed any Font, etc. constructor to the scripting side of things, as far as I'm aware.

...Actually, since my code was crappy, the only thing I did with the Font variable in particular was this:
Code: [Select]
var Font;

Then I proceeded to never use it. So I doubt it had any bearing on the actual existence of var Font.
The others were used, however, and that made it crash. In minisphere, I mean. :P
Title: Re: minisphere 1.2.2
Post by: Fat Cerberus on June 08, 2015, 12:15:35 pm
Speaking of crappy code, I discovered another bug thanks to that "Crap RPG" you posted a while back: minisphere's text rendering didn't handle tabs properly.  It tried to render the tab character directly instead of inserting spaces and didn't even honor it as a word break for purposes of word-wrapping.  It was also fixed in short order. :)
Title: Re: minisphere 1.2.3
Post by: Fat Cerberus on June 10, 2015, 01:17:01 am
minisphere 1.2.3 is out, with a bunch of fixes for bugs found by DaVince.
Title: Re: minisphere 1.2.3
Post by: mezzoEmrys on June 10, 2015, 01:44:49 am
Lots of fairly important bugs fixed, too, especially (for my current project) the hidden layer for obstructions one. I had just settled for fully transparent tiles, now i can actually make them visible in the editor :D
Title: Re: minisphere 1.2.3
Post by: DaVince on June 10, 2015, 05:21:17 am
Nice work. I love how quickly you always manage to fix these. :D

mezzoEmrys: if you find any bugs, please report them? The layer bug could have been fixed earlier if you had filed a bug, I would say. Lord English is really doing a great job with fixing these. :)
Title: Re: minisphere 1.2.3
Post by: Fat Cerberus on June 10, 2015, 08:25:01 am
I just realized I may be able to fix the "invalid base value" issue when variables are declared overwriting Sphere constructors.  I don't actually have to--it's a very bad practice I don't like to condone, but Duktape's error message for it is so obtuse that working around it is actually preferable at this point.

The issue arises because minisphere stores internal metadata properties in the object prototypes.  So when you declare a variable named, say, Font and then call a function which works with Font objects (e.g. GetSystemFont), the engine tries to access the Font prototype (this is why there is no line number--the error happened on the native side), which fails the same as if you did this in script:

Code: (javascript) [Select]
var Font; var foo = Font.prototype;


To elaborate on that: the boundary between native and script in Duktape is very thin.  Duktape's bytecode interpreter internally calls many of the same functions its public API does.  That's the reason why half of the Duktape API is macros and a big reason it can be as fast as it is.

Luckily Duktape also has this thing called the "stash" where you can store JS objects out of band such that they won't be garbage collected but are not otherwise accessible from script.  I could keep the object prototypes in the stash so that, if Font.prototype itself became inaccessible, the engine could still access it internally through the stash.  This would be enough to stop the errors, I think.
Title: Re: minisphere 1.2.3
Post by: Legaia on June 10, 2015, 11:09:34 am
Hey guys, glad to see work being done on new versions of the Sphere engine. I haven't played around with it yet, but does minisphere have all the same built in functions as Sphere has? IE. Can I follow old sphere tutorials or is there a new wiki/tutorial area specific to MiniSphere?
Title: Re: minisphere 1.2.3
Post by: Fat Cerberus on June 10, 2015, 11:12:20 am
minisphere is for all intents and purposes fully compatible with Sphere 1.5.  There may be a few obscure functions missing and the type checking for function parameters is stricter, but other than that anything you can do in one is done the same in the other.

minisphere also has a ton of new functionality not present in Sphere--you will have to look at minisphere-api.txt to find out how to use the new stuff.
Title: Re: minisphere 1.2.3
Post by: Fat Cerberus on June 10, 2015, 01:45:24 pm

I just realized I may be able to fix the "invalid base value" issue when variables are declared overwriting Sphere constructors.  I don't actually have to--it's a very bad practice I don't like to condone, but Duktape's error message for it is so obtuse that working around it is actually preferable at this point.


And... it's fixed as of the latest master commit. ;D
Title: Re: minisphere 1.2.3
Post by: FBnil on June 10, 2015, 04:48:45 pm
continue: FBnil was trying to install from source MiniSphere on his brand new Debian8 upgrade.

Ok, got it running.

Debian installs Allegro in /usr/lib/x86_64-linux-gnu/ and those were 5.0.10 and overruled the 5.1.10, so after a
Code: [Select]
apt-get remove liballegro5.0:amd64
it finally compiled! Checking the fog demo... nice work! However, still buggy. You see, the fog layer also needs to stay in sych with the background layer, so if I move, it is not scrolling in respect of the player, but of the other background.
So starting the fogdemo, when I move left/right, the fog seems to be following the player, instead of following the ground.

MiniSphere Source is clean and understandable. Still need to peruse it to commit an index map into memory. I like what I have seen so far! Kudo's Bruce!

I am impressed with duktape. At under 3Mb uncompressed source, compared to the 20Mb bz2 compressed tarball from spidermonkey.

edit: Also, duktape has been updated: 2015-06-02   duktape-1.2.2.tar.xz   maintenance release (http://duktape.org/download.html)

1.2.2 (2015-06-02)

    Fix harmless MSVC warnings when using DUK_OPT_FASTINT on x86 (GH-172)
    Fix compile error from array fast path when using low memory options (GH-174)
    Fix harmless MSVC warnings for size_t casts on x64 (GH-177)
    Fix potential NULL pointer dereference in duk_is_dynamic_buffer() and duk_is_fixed_buffer() when index is outside of value stack (GH-206)


Oh noes! No ColorMatrix objects!
Title: Re: minisphere 1.2.3
Post by: mezzoEmrys on June 10, 2015, 05:46:47 pm

The layer bug could have been fixed earlier if you had filed a bug, I would say.


To be honest, I didn't realize it was a bug at first, I thought it seemed intuitive enough that it did that. Didn't even think to check it on 1.5, since I was writing a game from scratch targeted at minisphere anyway. It wasn't until I saw your issue that I thought to check it.
Title: Re: minisphere 1.2.3
Post by: Fat Cerberus on June 10, 2015, 05:54:12 pm
 @FBnil
I'm pretty sure minisphere should render the fog the same as Sphere 1.5.  At least it looked that way in my tests.  Are you sure you have the latest master from GitHub?  I'm also already using Duktape 1.2.2.  If the copy in your source is older, you have an outdated minisphere build.

As for ColorMatrix objects, I'll add those to the list for the upcoming 1.3 release.  I knew I was missing something!
Title: Re: minisphere 1.2.3
Post by: DaVince on June 10, 2015, 07:12:22 pm
I haven't been as excited about Sphere in general as I am now. Getting back into the swing of things. Once vacation starts, I'm going to enjoy figuring out those new functions and making a game!
Title: Re: minisphere 1.2.3
Post by: Radnen on June 10, 2015, 10:56:55 pm
What are we doing wrong with OpenGL? I knew there's something messed up about how we are implementing OpenGL, because if you think about it, it can be used to display vast fully-realized high fidelity 3D scenes in 60fps, but we can barely make it all of our engines run more than 20000+ 2D entities on screen at the same time, quickly.

Take for instance this stupid WebGL test, (it's by no mean 'stupid'): http://www.goodboydigital.com/pixijs/bunnymark/

I can render 200000 - two hundred thousand - dancing bunnies and still got 60fps out of the rendering window!! If there are 4 vertex points per object, then we have 800000 verticies, which is still less than a typical 3D scene has on screen. A good PC can handle billions of vertices per second. But, let's say your computer can handle 1 billion verts on screen per second, that makes 800000 verts take only .0008s or .8ms, plenty of room to handle 16.67ms of time per frame. So of course this runs at 60fps. I'm just shocked we can't get this performance out of our native engines. Is Chrome's V8 coupled with WebGL really that damn fast?

I'm doing everything in the book right with my SphereSFML. I'm sending 1 batch of vertices to the pipeline, with a single base texture (atlas, sprite batch, etc.) and I can't get anywhere close to 60000 at 60fps, let alone 200000! And I'm doing some basic logic rather than just dead, simple images (I'm rotate blitting them).
Title: Re: minisphere 1.2.3
Post by: Fat Cerberus on June 10, 2015, 11:33:35 pm
I tested the bunny thing, I got to about 20,000 bunnies before the framerate started dropping off, and it actually started to hiccup before that, around 10,000.  Of course this is with the iGPU in my AMD A10-6800K, so I'm sure I could get more bunnies with an actual graphics card.  Full disclosure: Firefox.
Title: Re: minisphere 1.2.3
Post by: Fat Cerberus on June 10, 2015, 11:36:07 pm

Oh noes! No ColorMatrix objects!


See here:
https://github.com/fatcerberus/minisphere/commit/a14d2b2bd0c6c2648634b141587cc91c5178d93a

Everything is now implemented except for surface.applyColorFX4() because I haven't figured out how to bilinear-interpolate the matrices yet.  I'll have to look at the Sphere 1.6 source again.

Edit: Holy crud, ApplyColorFX4() is a beast of a function, I have no clue what it's even doing!  It does some crazy interpolation not only on the matrices, but on the pixels themselves.  applyColorFX() was easy, but this... well, remember how I said I wouldn't use code I don't understand?  Yeah, this is a perfect example. :P  I will have to study it a bit more, I'm not just going to paste it in.
Title: Re: minisphere 1.2.3
Post by: Radnen on June 11, 2015, 12:48:57 am

I tested the bunny thing, I got to about 20,000 bunnies before the framerate started dropping off, and it actually started to hiccup before that, around 10,000.  Of course this is with the iGPU in my AMD A10-6800K, so I'm sure I could get more bunnies with an actual graphics card.  Full disclosure: Firefox.


On chrome I got 200000(60fps), on Firefox I get 200000(50fps). I notice the test dips more under Firefox, meaning Chromes V8 is indeed faster in a real example or they have a more optimized WebGL context(?). I have a really nice Nvidia GPU, the GTX970 which didn't cost me a whole lot compared to the 980 or the 980ti (which is an inane amt of cash for such little more power1).

It's just weird, that's all. It might be the overhead, and I think it's the Sphere API calls. I think web has a better, leaner link between the OpenGL and the JS.

1. http://www.newegg.com/Product/Productcompare.aspx?Submit=ENE&N=100007709%20600536049&IsNodeId=1&bop=And&CompareItemList=48%7C14-487-087%5E14-487-087-TS%2C14-487-142%5E14-487-142-TS%2C14-487-088%5E14-487-088-TS&percm=14-487-088%3A%24%24%24%24%24%24%24
Title: Re: minisphere 1.2.3
Post by: Fat Cerberus on June 11, 2015, 12:53:18 am
The Sphere API is probably it.  A JS function call is much, much more expensive than a native call no matter how you slice it, for a lot of reasons.  arguments object creation is a big one - this document from the Duktape repo was very enlightening and an interesting read:
https://github.com/svaarala/duktape/blob/master/doc/arguments-object.rst
Title: Re: minisphere 1.2.4
Post by: Fat Cerberus on June 11, 2015, 12:56:25 am
I decided the "invalid base value" fix was too important to hold it back until 1.3 and backported it.  So, well... minisphere 1.2.4 is available for download. :)
Title: Re: minisphere 1.2.4
Post by: FBnil on June 11, 2015, 04:30:28 am
Thanks L.E. did a git pull. Now my techdemo complains that the code is not "strict" enough (back to that then, make all my libs compatible with MS).
And thanks for looking at those ApplyColorFX4() is useful though, as a crud way to emulate lights and shadow.
I do remember there was an explanation for it somewhere. Do you even sleep? I sleep and there are so many messages.

Meanwhile, I used tcc to compile some of the examples of duktape. Then compress them with upx. tcc has its limitations, gcc -os (compile and make the smallest size) is still way better at making smaller files.

"Drunken Wee master" needs BezierCurve .... was that even a function? ... yes... yes it was...
"surface.bezierCurve(c,step,x1,y1,x2,y2,x3,y3,x4,y4) ". I do not even remember these functions...


@Radnen: Maybe you create and destroy object calls each loop? Maybe you require a table where you keep them alive each loop?

@DaVince: Yay! new game from DaVince comming up!
Title: Re: minisphere 1.2.4
Post by: Fat Cerberus on June 11, 2015, 06:28:14 am
I have... odd sleep patterns.  And rest assured applyColorFX4() is getting implemented.  I figured it out once I started reimplementing it what it was doing, it's simple bilinear interpolation on the individual cells of the 4 matrices.  The way the code is written in Sphere 1.x doesn't make that obvious though.  Needless to say, my version will be much better annotated. :D

Not sure what that "not strict enough" thing is about... Sounds weird.

Bézier curves might be more difficult though.  The thing is, I don't really understand the math behind them and Allegro doesn't (to my knowledge) have a function built in to draw them.

Edit: As of about 5 minutes ago, applyColorFX4() is implemented.  I'm thinking my implementation may not be the fastest thing ever, Sphere does some weird thing with bitshifts that I don't quite understand, and I use fmin/fmax to clamp the pixels, which is probably a massive performance hit (doing int->float->int several times for every pixel has to take a toll).  The base code should be very easy to follow though:
https://github.com/fatcerberus/minisphere/blob/1fe34c2d603a5c49613db212c53ca8a1d3e2ccb7/src/image.c#L318-L381
Title: Re: minisphere 1.2.4
Post by: DaVince on June 11, 2015, 08:15:38 am
The BunnyMark only gets me as far as ~35000 bunnies before the FPS starts dropping, but then again I'm on a notebook with an NVIDIA GT540m and a 2Ghz Sandy Bridge CPU, so not exactly the most exciting thing.
For me, as long as games perform well, I'm happy, and in that regard minisphere is doing a lot better than original Sphere. (There is a level in Sir Boingers that starts lagging out once I add only a few more cupcakes; minisphere uses about 50% on a single thread at that point and 17% more on another.)

Quote
@DaVince: Yay! new game from DaVince comming up!

Whoa. I'm humbled. :> Now how about you make one again too? I liked your old games and demos. :)
Title: Re: minisphere 1.2.3
Post by: Flying Jester on June 11, 2015, 11:43:39 am

What are we doing wrong with OpenGL? I knew there's something messed up about how we are implementing OpenGL, because if you think about it, it can be used to display vast fully-realized high fidelity 3D scenes in 60fps, but we can barely make it all of our engines run more than 20000+ 2D entities on screen at the same time, quickly.


Three really important things I know I don't do are instancing, geometry/compute shaders (or Cuda/CG interop), and LOD.

Either way, TurboSphere can push 50,000 shapes and stay at 60(ish) FPS (So still more efficient than the BunnyMark on my machine). More or less vertices, depending on how they are divided between shapes and groups. I would highly suspect that's because I use OpenGL 4, which means that the single Group.draw() call to draw all those shapes results in only a fraction that many OpenGL calls. This is why the Galileo API is the way it is. It stops you from needing thousands of calls for thousands of sprites. That's also why I dropped the old API altogether. It's really not how OpenGL or D3D or any modern rendering technology works, and it shows.

But we will always be slower than dedicated 3D engines. They will always have specific functions built in, which the engine can be optimized for at the cost of flexibility. I'm more content to lose a little performance be be generally useful :)
Title: Re: minisphere 1.2.4
Post by: Fat Cerberus on June 11, 2015, 12:45:21 pm
I just added ByteArray inflate/deflate support:
https://github.com/fatcerberus/minisphere/commit/40063460804b7834b18b0e71099fab44184f7310

Used like:
var little = bytearray.deflate([level]);
var big = bytearray.inflate();

Honestly though, I'm not sure how useful this is in a game engine - I only added it because 1) Sphere 1.6 has it and 2) I already had a zlib dependency thanks to libmng, so may as well use it.
Title: Re: minisphere 1.2.4
Post by: Flying Jester on June 11, 2015, 01:35:30 pm
It would be useful saving data or sending data across a network.

zlib is so fast and so common that there's not really a good reason not to use it when sending or saving more than a few kilobytes of data.

You also pulled in zlib once you supported png.
Title: Re: minisphere 1.2.4
Post by: FBnil on June 11, 2015, 05:01:08 pm
Works perfectly:
Code: [Select]
function game(){
var originaltext = "Hello Sphere Worlds!";
var source = CreateByteArrayFromString( originaltext );

var compressed = DeflateByteArray( source , 9); // Sphere Syntax
//var compressed = source.deflate(9); // MiniSphere Syntax

if( compressed !== null ){
var inflated = InflateByteArray(compressed, 1024*16); // Sphere Syntax
//var inflated = compressed.inflate(); // MiniSphere Syntax
if(inflated !== null){
var copy = CreateStringFromByteArray(inflated);
if(copy === originaltext)
  Abort( "SUCCES:" + CreateStringFromByteArray(inflated) );
else
  Abort("ByteArray to string failed");
 
}else{
Abort( "Decompression failed." );
}
}else{
Abort("Compression failed.")
}
}

// MiniSphere to Sphere syntax

function DeflateByteArray(BA_obj,ratio){
  return BA_obj.deflate(ratio);
}

function InflateByteArray(BA_obj,size){
  return BA_obj.inflate(size); // need size or we can make a inflate bomb?
}


edit: A deflate bomb is possible:  http://security.stackexchange.com/questions/51071/zlib-deflate-decompression-bomb
Title: Re: minisphere 1.2.4
Post by: Fat Cerberus on June 11, 2015, 05:03:59 pm

You also pulled in zlib once you supported png.


In theory, yes.  In practice no because Allegro uses GDIplus to load images on Windows.  Nonetheless, I did pull it in with libmng, so your point stands. :)
Title: Re: minisphere 1.2.4
Post by: Fat Cerberus on June 11, 2015, 05:05:14 pm

Works perfectly:
Code: [Select]
function game(){
var originaltext = "Hello Sphere Worlds!";
var source = CreateByteArrayFromString( originaltext );

var compressed = DeflateByteArray( source , 9); // Sphere Syntax
//var compressed = source.deflate(9); // MiniSphere Syntax

if( compressed !== null ){
var inflated = InflateByteArray(compressed, 1024*16); // Sphere Syntax
//var inflated = compressed.inflate(); // MiniSphere Syntax
if(inflated !== null){
var copy = CreateStringFromByteArray(inflated);
if(copy === originaltext)
  Abort( "SUCCES:" + CreateStringFromByteArray(inflated) );
else
  Abort("ByteArray to string failed");
 
}else{
Abort( "Decompression failed." );
}
}else{
Abort("Compression failed.")
}
}

// MiniSphere to Sphere syntax

function DeflateByteArray(BA_obj,ratio){
  return BA_obj.deflate(ratio);
}

function InflateByteArray(BA_obj,size){
  return BA_obj.inflate(size); // need size or we can make a inflate bomb?
}



You don't even need the shims with the latest commit, I added the Sphere compatible functions.  :)

Could you also test the ColorMatrix support?
Title: Re: minisphere 1.2.4
Post by: FBnil on June 11, 2015, 05:18:30 pm
sure thing, will work on it over the weekend.
Title: Re: minisphere 1.2.4
Post by: Fat Cerberus on June 11, 2015, 05:22:31 pm
One potential pitfall: minisphere will throw if an inflate/deflate fails, apparently Sphere returns null instead?

Good point on the potential for inflate bombs though, I'll add an optional size parameter when I get a chance.
Title: Re: minisphere 1.2.4
Post by: DaVince on June 11, 2015, 05:50:48 pm
Didn't original Sphere never throw exceptions because implementing exceptions was a difficult mess, or am I wrong on that? Even if it did though, it'd be pretty inconsistent with everything else. I've never seen try..catch in Sphere...
Title: Re: minisphere 1.2.4
Post by: Flying Jester on June 11, 2015, 05:57:45 pm

Didn't original Sphere never throw exceptions because implementing exceptions was a difficult mess, or am I wrong on that? Even if it did though, it'd be pretty inconsistent with everything else. I've never seen try..catch in Sphere...


I thought it did? For instance, you could try...catch a LoadImage in case the image didn't exist?

Canonical Sphere scripts certainly don't use exceptions much at all, either way.
Title: Re: minisphere 1.2.4
Post by: Fat Cerberus on June 11, 2015, 06:03:42 pm
Yeah, LoadImage et al will throw on error.  sphere APIs that return null on failure are in the minority I think.
Title: Re: minisphere 1.2.4
Post by: DaVince on June 11, 2015, 06:13:55 pm
Huh. Well, how about that. I was certain there was a whole issue around this, but guess not.

...And here I have this whole script/function dedicated to loading resources and doing its best determining whether a resource exists or not and providing placeholders for years... Time to update that script to make use of it, I suppose.
Title: Re: minisphere 1.2.4
Post by: Flying Jester on June 11, 2015, 06:18:34 pm
I'm 99% certain that anytime you get the black screen + white text that blames a SS_* function, you could have used a try...catch on it :)

Just for some reason, none of us ever did that...
Title: Re: minisphere 1.2.4
Post by: Fat Cerberus on June 11, 2015, 06:39:52 pm
On that note, the minisphere error screen is so much nicer looking than the Sphere 1.5 one.  Just have to toot my own horn a little... ;)
Title: Re: minisphere 1.2.4
Post by: Flying Jester on June 11, 2015, 06:40:53 pm

On that note, the minisphere error screen is so much nicer looking than the Sphere 1.5 one.  Just have to toot my own horn a little... ;)


But can you select the text on the error screen?

You can select the text in TurboSphere's error screen :)
Title: Re: minisphere 1.2.4
Post by: Fat Cerberus on June 11, 2015, 06:41:56 pm
I wanted to do something like that, but as Allegro has no clipboard routines...
Title: Re: minisphere 1.2.4
Post by: Flying Jester on June 11, 2015, 06:51:08 pm
The way I did it was first using SDL's dialog methods, and then later a little platform-specific code to make dialogs for X11, OS X, and Win32.
Title: Re: minisphere 1.2.4
Post by: DaVince on June 11, 2015, 07:15:13 pm
I think that if you do implement error message copying, you could get away with simply copying the message to the clipboard once you click the screen.
But then again, the errors output to the terminal too, so how necessary is it really? Well, useful for reporting the errors, I suppose. :P
Title: Re: minisphere 1.2.3
Post by: Radnen on June 11, 2015, 08:08:55 pm
This is why the Galileo API is the way it is. It stops you from needing thousands of calls for thousands of sprites. That's also why I dropped the old API altogether. It's really not how OpenGL or D3D or any modern rendering technology works, and it shows.

But we will always be slower than dedicated 3D engines. They will always have specific functions built in, which the engine can be optimized for at the cost of flexibility. I'm more content to lose a little performance be be generally useful :)


I think you're right about the API. Sphere's immediate drawing routines are the issue here, even if you do creative batching like in SphereSFML. There are newer tricks that are easier to use had Sphere's API been different, which is the case for Galileo.
Title: Re: minisphere 1.2.4
Post by: Fat Cerberus on June 12, 2015, 12:48:05 am
Agreed.  However I won't drop the legacy routines as they serve as a convenient stepping stone.  Don't underestimate the value of having a fully Sphere-compatible engine.  It makes porting painless, so then you get access to all the nifty new features (like Galileo!) pretty much for free.  And then eventually you do start using them, and before long you wonder how you ever lived without the neat new additions. ;D

Basically minisphere is set up to let you learn new ways of doing things at your own pace rather than being forced into it, which from my own past experience I can say tends to make you resent "the new way".  Better to be able to ease your way into things than being thrown to the wolves.

That said, while the legacy primitives are included in the engine, I deliberately didn't document them for precisely the reason Jester mentioned.  They are available for backwards compatibility and easy porting, but for all intents and purposes they should be considered deprecated at this point.
Title: Re: minisphere 1.2.4
Post by: Fat Cerberus on June 12, 2015, 01:19:53 am

edit: A deflate bomb is possible:  http://security.stackexchange.com/questions/51071/zlib-deflate-decompression-bomb


Today I added an optional max_size parameter to bytearray.inflate() and InflateByteArray(), to defend against inflation bombs.  I left the default dynamic allocation behavior in for when a max size isn't provided however, as that may sometimes be preferable for trusted data, such as local save files.  Any inflation of data coming over a socket should definitely include a max size clause, though, I totally agree with that.

minisphere 1.3 is nearing release, I just want to do some updates to miniRT and then it should be good to go.  Unless I decide to add more features, I swear there was another new feature idea I had but now I've forgotten it...
Title: Re: minisphere 1.2.4
Post by: FBnil on June 12, 2015, 04:25:11 am
I had to create a shell function "gup", to update my git each time... man... chose changes are pumping through faster than I can make testcases.
Title: Re: minisphere 1.2.4
Post by: Fat Cerberus on June 12, 2015, 09:08:15 am
Yeah, nobody who saw this engine would believe it was one guy who put it together in 3 months, huh?  Nonetheless... ;D

I started development towards the end of January, minisphere 1.0 was released mid-to-late April.

It's funny, I'm generally bad at finishing projects I start, but minisphere... I don't know, it's been very easy for me to keep the wheels of development turning with this project for some reason.  Which is a good thing of course, I just don't understand it. :P

Title: Re: minisphere 1.2.4
Post by: Fat Cerberus on June 12, 2015, 11:49:07 am
I remembered the feature I wanted to add: SPK support!  I always aim to have at least one killer new feature for each version bump, so 1.3's will be archives.  For game packages I want to support, at the very least, SPK and ZIP.  To do this, there's apparently a zlib-licensed library called "PhysicsFS" that will let me access files directly from archives:

http://icculus.org/physfs/
Title: Re: minisphere 1.2.4
Post by: DaVince on June 12, 2015, 01:23:38 pm
Awesome! I have some ideas for redistributable games in that regard:

- Obviously, direct filetype association with spk files would be nice. Double-click the SPK, play the game.
- Renaming the zips with a different extension (like .spk2 or .spherezip or something). This makes filetype association easier for Sphere zips. (Compare to .cbz "comic book zip" files, which are just archives filled with pictures)
- The game launcher detecting these packages if you put them in games/, of course. :)

And some more special ideas:
- If you distribute just the engine binary + an spk file, it'll pick up on the spk file so you don't even need a games or startup folder, it'll just open that when you start the engine.
- The ability to load multiple spk/zip files. Could be useful for loading the game and then loading extra resources on top of it (like a mod, a levelpack, etc.)
Title: Re: minisphere 1.2.4
Post by: Fat Cerberus on June 12, 2015, 01:57:43 pm
Since minisphere reports as Sphere 2.0 now, we can definitely make an .spk2 format.  I think we'd have to standardize on an archive format though.  Zip seems the best option, as it can be supported easily enough with just zlib, regardless of what minisphere ends up using to support it.  Plus it seems to be what most compressed formats go with.  OpenDocument and Office 2007+ docx files are just relabeled zips, for example.

7z would of course compress better, but would hinder performance as decompression can be slow, especially if solid compression is used.  Ideally I'd want a format with decent compression but without sacrificing too much random access performance.  A non-solid deflate-based format like zip fits the bill, I'd think.

Where things get hairy is handling write access.  RawFile is the biggest issue here--the same API is used to open a file for either reading or writing, but you can't really write to a package at runtime.  It might be necessary for me to step back and look at refactoring the asset management first, before baking in package support.

I would also have to make sure running a game from a package is transparent.  This is an issue I know Sphere has had: some things just don't work when running games from an SPK.  It's a big reason I gave up on packaging games back when I was using 1.5.  Which is a shame, because being able to distribute a single .spk for your game is nice for when the person on the other end may not really know what to do with a zip file.

You know, I'm actually beginning to wonder if the FS sandbox imposed by the original engine wasn't deliberate, but just a necessary evil of supporting packages.  Not to say you can't dispense with the sandbox by using fallback clauses, but for consistency's sake Chad Austin might have decided to just sandbox always, regardless of the packaging method or lack thereof.
Title: Re: minisphere 1.2.4
Post by: Flying Jester on June 12, 2015, 02:36:36 pm

Since minisphere reports as Sphere 2.0 now, we can definitely make an .spk2 format.  I think we'd have to standardize on an archive format though.  Zip seems the best option, as it can be supported easily enough with just zlib, regardless of what minisphere ends up using to support it.  Plus it seems to be what most compressed formats go with.  OpenDocument and Office 2007+ docx files are just relabeled zips, for example.

7z would of course compress better, but would hinder performance as decompression can be slow, especially if solid compression is used.  Ideally I'd want a format with decent compression but without sacrificing too much random access performance.  A non-solid deflate-based format like zip fits the bill, I'd think.


I believe that a combination of tar or cpio and zlib or xz is the best. tar and cpio provide the same strengths, but are far simpler.
The reason I never got around to fully supporting spk's was that writing to a file from a game that is running from one is very complicated, as you mention. I finally said enough, and gave up on packages, since at that point the benefits seemed very small for the amount of complexity they introduced.

My final plan is to just unpack the package to some local directory and run the game there. If it looks like I've done that before, I merge the existing directory with what I unpack, favoring existing content. This bypasses many of the issues associated with packages, at the expense of having both a packaged and unpackaged game on the local disk at once.
Title: Re: minisphere 1.2.4
Post by: Fat Cerberus on June 13, 2015, 01:51:46 am
In the interest of making package support as painless to implement as possible, I've decided it would be best to standardize on a file system.  I call my initiative SphereFS, and it'll be a key pillar of the Sphere 2.0 API.  More details forthcoming, but suffice to say it will be backwards compatible with the Sphere 1.x sandbox but would allow, e.g. absolute paths via an escape.

SphereFS support will have its own extension designation in minisphere, sphere-spherefs.  Once I've fleshed things out some more, I will post a new thread in the Engine Development forum.

where doing it man
where MAKING THIS HAPEN
Title: Re: minisphere 1.2.4
Post by: N E O on June 13, 2015, 11:55:25 pm
Re Bezier curves - Fortunately for you I understand the maths and once wrote a script to use in Sphere. It's in the Wayback of the old wiki somewhere. I also C++-ed it for my hello-phoenix project (https://github.com/apollolux/hello-phoenix/blob/master/hello/ncurve.hpp) on my GitHub. The de Casteljau curve algorithm is essentially a divide-and-conquer one, but you may also need to limit the recursion depth if you don't check for point distance (NCurve in hello-phoenix does both, but I think the JS version only tests point distance).

Re spk/zip - If you choose to implement reading of renamed zip files as the next generation of spk and you want to keep a three-character extension, I recommend spz. If it's renamed 7z, sp7.

Re SphereFS - Feel free to make a "Future of Sphere" post for it, maybe including spitballing sphere:// or sph:// protocol handling or something too for a bit farther into the future.
edit: seems you already posted, I didn't finish catching up +neo
Title: Re: minisphere 1.2.4
Post by: Fat Cerberus on June 14, 2015, 03:29:33 am
Ugh, the package support wasn't even the most painful part of the SphereFS implementation, it's the abstraction that's the killer.  The whole engine was written with the assumption files will exist on the local FS, forcing me to not only have to implement custom sfs_fopen() etc. but even things like GetFileList() ended up getting broken.

The biggest issue though is loading images and such through Allegro.  If it weren't for Allegro's memfile add on (file-mapped memory!), a clean abstraction wouldn't even be possible--I'd have to resort to temp files.
Title: Re: minisphere 1.2.4
Post by: Fat Cerberus on June 14, 2015, 11:25:45 am
...nonetheless, I finally got it up and running this morning. :) A few things, like MNG loading, are still broken, but for the most part the engine is back in working order and will run a game from either a local folder or an SPK.

SPK is implemented by an spk_fopen() function which unpacks the requested file into memory and an emulated spk_fread() etc. which operate on the buffer.  This avoids a potentially lengthy unpacking step on startup without sacrificing too much performance, as memory is fast so the only additional overhead over straight disk access is the inflate step.

The end result is that the rest of the engine doesn't know or care where or how the file is stored, it just gives SphereFS a sandbox pointer and a filename and gets a file handle in return.  It's a very clean setup this way. :D

edit: It actually might be possible to support writing to an SPK, now that I know how the format is set up.  Every file, and the index itself, has an explicit offset stored in the headers, which means it's trivial to just append a new or modified file to the end.  For existing files you just modify the index entry, and new files can be tacked onto the end.  Of course this could cause the SPK to grow inordinately large without some method of compaction available... Just a thought.
Title: Re: minisphere 1.2.4
Post by: Fat Cerberus on June 15, 2015, 02:52:24 am

Awesome! I have some ideas for redistributable games in that regard:

- Obviously, direct filetype association with spk files would be nice. Double-click the SPK, play the game.
- Renaming the zips with a different extension (like .spk2 or .spherezip or something). This makes filetype association easier for Sphere zips. (Compare to .cbz "comic book zip" files, which are just archives filled with pictures)
- The game launcher detecting these packages if you put them in games/, of course. :)

And some more special ideas:
- If you distribute just the engine binary + an spk file, it'll pick up on the spk file so you don't even need a games or startup folder, it'll just open that when you start the engine.
- The ability to load multiple spk/zip files. Could be useful for loading the game and then loading extra resources on top of it (like a mod, a levelpack, etc.)


Going to take these one at a time.  Now that the SPK support is actually implemented, I can tackle these.

- Filetype association is best left to an installer, methinks.  Since the official distribution is for Windows, I think I can handle that.  minisphere 1.3 will be the first to be distributed as an installer.  That is, assuming Google will let me upload it to Drive, I think it might block uploading .exe files?  But yes, association with SPK would be nice.

- I'm thinking zip support won't make it into 1.3 after all.  Having actually implemented SPK and seen the inner workings of the format, it seems more than adequate for its intended purpose.  Plus it's insanely fast: I saw literally no performance difference at all running Specs from an SPK vs. directly.  They are even fast to compress, too--the Sphere 1.5 editor made a 35MB SPK out of my Specs project folder in seconds!  This to me is a clear case of "if it ain't broke, don't fix it" :P

- Self-evident, skipping. :)

Now, the "special" ideas:

- This is an awesome idea and not too difficult to implement either.  Only one question: If both a single SPK and a startup folder are present, which one takes precendence?

- This adds extra complexity to the engine, probably out of the scope of the 1.x releases.  A good idea for minisphere 2.0, though, I'll add it to the GitHub issue for that. :D
Title: Re: minisphere 1.2.4
Post by: DaVince on June 15, 2015, 05:39:16 am
Quote
- This is an awesome idea and not too difficult to implement either.  Only one question: If both a single SPK and a startup folder are present, which one takes precendence?

I'd say the startup folder, because that folder existing is evidence the entire package was not made to be a standalone game. Perhaps the game could still appear in the games list; I believe the default startup game just starts the game if it's the only one it finds anyway.

As for the special ideas: they're just ideas. But it's awesome that you're tackling both of them anyway. :P
Title: Re: minisphere 1.2.4
Post by: Fat Cerberus on June 15, 2015, 02:40:29 pm
I just enabled write access for games running from packages.  Actually modifying the SPK file at runtime would be time-consuming to implement and the file isn't guaranteed to be writable anyway, so if a game requests write access to an existing file, it is saved in ~/Sphere/.spkcache/[filename.spk] (the Linux meaning of ~, not Sphere's :) )

Note that this differs from Sphere 1.5, which puts the cache in packages under the engine.  Since nowadays we have to assume the engine directory is read-only, this needed to be changed.
Title: Re: minisphere 1.2.4
Post by: N E O on June 15, 2015, 07:48:46 pm
Re non-local files - For now, I recommend keeping all the various Sphere format file I/O functionality focused on local files. For non-local files, we can certainly consider expanding the network functions' capabilities to include a cURL implementation/shim or something as sugar for the existing socket functions for non-vanilla pre-2.0 engines and eventually fold them into official 2.0 API. Once the kinks are worked out, this folding in would also eventually allow non-local file reading on the Sphere format file I/O.

Then again, if it turns out to be trivial to change file I/O in minisphere to allow at least networked file I/O (as in less than 2 hours or so to add) feel free to add it now, show us some working tests, and we can have it much sooner. I'll then fully support such an endeavor as marked to be made official ASAP.
Title: Re: minisphere 1.2.4
Post by: Fat Cerberus on June 16, 2015, 03:35:31 am

Then again, if it turns out to be trivial to change file I/O in minisphere to allow at least networked file I/O (as in less than 2 hours or so to add) feel free to add it now, show us some working tests, and we can have it much sooner. I'll then fully support such an endeavor as marked to be made official ASAP.


The beauty of SphereFS is that it canonizes the Sphere 1.x sandbox.  For example when you call LoadImage("foo.png") under minisphere 1.3, you are not requesting <game path>/images/foo.png, the canonical name of the asset is literally:
images/foo.png

And names of this form are what get passed around inside the engine, not actual file paths.  SphereFS itself handles the final path resolution and I/O behind the scenes.  So yes, network I/O would be very easy to implement.  The tricky part is the synchronicity--the default APIs for loading assets are blocking in nature, which would slow them down running over the wire.

Long story short, the actual network file I/O isn't the issue, it's the API.
Title: Re: minisphere 1.2.4
Post by: Fat Cerberus on June 16, 2015, 02:52:16 pm
As of the latest master, minisphere will auto-launch a single .spk file placed alongside the engine executable, making game distribution drop-dead simple.  I just have to work out a couple more glitches and 1.3 should be ready for release.  ;D
Title: Re: minisphere 1.3.0
Post by: Fat Cerberus on June 17, 2015, 12:51:17 pm
minisphere 1.3 is out! 8)

This brings Sphere 2.0 SphereFS support, and with it the ability to run games packaged as SPKs!  You'll note that the startup game was converted to an .spk... ;)

As is usual for an x.x.0 release, expect bugs to show up.  So download minisphere 1.3 and test away!
Title: Re: minisphere 1.3.0
Post by: Fat Cerberus on June 17, 2015, 08:47:33 pm
Oh, not mentioned in the changelog, but minisphere 1.3 is fully Unicode-aware under the hood.  Person names, etc. are all UTF-8 internally, so you can feel free to use any characters you want in them and the engine won't choke on it. :)
Title: Re: minisphere 1.3.0
Post by: DaVince on June 18, 2015, 08:18:56 am
Nice work, as usual! ;D
Title: Re: minisphere 1.3.1
Post by: Fat Cerberus on June 18, 2015, 01:23:23 pm
minisphere 1.3.1 is available, with a ton of fixes for regressions introduced in 1.3.0.  This is my biggest maintenance release in a while, so be sure to check the changelog. :D

@FBnil: The fog demo should now work properly as of this release.
Title: Re: minisphere 1.3.1
Post by: FBnil on June 18, 2015, 01:38:02 pm
yes yes! it works now!

Tested ApplyColorFX(). (test script added).
Same behavior as Sphere 1.6, however, as you stated earlier, much slower.

Still have to modify the unittests, and publish it.
Title: Re: minisphere 1.3.1
Post by: Fat Cerberus on June 18, 2015, 01:51:21 pm
I think part of the speed problem is that I pass colormatrix_t objects around by value, so a lot of copying goes on, even more so for applyColorFX4().  But I also know Sphere does clever tricks with bitshifts for this operation, which I'm sure speeds it up considerably.  minisphere also calls a function to do the matrix interpolation, so that adds even more overhead if the compiler doesn't inline it.

I purposely didn't optimize it for this initial implementation, though.  I wanted to make sure I understood the theory behind it first.  I know now that it's just simple bilinear interpolation, but it wasn't easy to figure that out with all the magic tricks going on. :)
Title: Re: minisphere 1.3.1
Post by: Flying Jester on June 18, 2015, 01:55:44 pm
minisphere also calls a function to do the matrix interpolation, so that adds even more overhead if the compiler doesn't inline it.


Which is probably at least an order of magnitude smaller than passing an object on the stack :) What is the sizeof for a colormatrix_t?
Title: Re: minisphere 1.3.1
Post by: Fat Cerberus on June 18, 2015, 01:57:56 pm
int is 32 bits on Windows, a colormatrix_t has 12 of them.  So 48 bytes, or 96 bytes on a platform with 64-bit ints.

Edit: Honestly I'm starting to think most of the performance hit is that locking a bitmap in Allegro is just insanely expensive for no apparent reason.  I could understand it when I was using hardware textures, but switching surfaces to software offered no real performance improvement in, e.g. getPixel() until I also added the pixel cache for that.
Title: Re: minisphere 1.3.1
Post by: Fat Cerberus on June 19, 2015, 12:45:58 am
So I ran the profiler on minisphere to figure out what was causing the massive slowdown on my battle screen, and came back with some... interesting results.

As you can see on the Summary page, the function causing the majority of the bottleneck is js_Font_drawText().  No surprise there, I already knew minisphere was slow at rendering text.  However, the runner-up shocked me: js_new_Color, the Color constructor.  You can see the breakdown for that in the second screenshot.  Apparently creating Color objects on demand on expensive!

For good measure, the third screenshot shows the main culprit in the text rendering bottleneck: al_hold_bitmap_drawing(false);  This should be ridiculously fast, as fonts are atlased so Allegro can just send all the vertices at once.  But apparently not...
Title: Re: minisphere 1.3.1
Post by: Flying Jester on June 19, 2015, 01:59:17 am
I found that the color constructor adds noticeable overhead in TS, too. My main advice is just to not create colors on the fly as much as possible.


For good measure, the third screenshot shows the main culprit in the text rendering bottleneck: al_hold_bitmap_drawing(false);  This should be ridiculously fast, as fonts are atlased so Allegro can just send all the vertices at once.  But apparently not...


I'd be curious how the Turbo runtime's font renderer runs in Minisphere.
Title: Re: minisphere 1.3.1
Post by: Fat Cerberus on June 19, 2015, 02:46:03 am
I still can't get minisphere to load the Turbo runtime successfully due to all the circular RequireScripts.  How do you have RequireScript implemented in TS?  I evaluate the script AFTER entering it into the tracking list, to avoid infinite circular recursion.  So what ends up happening for me is that, after requiring map_engine.js, at some point in the require chain it gets to segment.js and Turbo is undefined because it's still in the middle of resolving the RequireScript chain.
Title: Re: minisphere 1.3.1
Post by: Flying Jester on June 19, 2015, 03:41:35 am
https://github.com/FlyingJester/TurboSphere/blob/master/src/configmanager/openscript.cpp#L68-L104 (https://github.com/FlyingJester/TurboSphere/blob/master/src/configmanager/openscript.cpp#L68-L104)

...Which leads to large stacks at certain points...

That's very strange. In TurboSphere, RequireScript enters a script into the list of evaluated scripts immediately after it checks if it was previously evaluated, which is right before it evaluates the next script.

But every one of the runtime's scripts should have the declaration of the Turbo object before anything that uses it--you should be able to require any one of them, and in any order, and not have that issue. It seems odd to me that Minisphere would complain about Turbo being undefined, unless the RequireScripts are not sharing all declarations until after they have completed or something similar. Or somehow our implementations are really that different? I think we do it the same way, and I can't really think of another way that would even come close to working properly.
Title: Re: minisphere 1.3.1
Post by: Fat Cerberus on June 19, 2015, 09:23:31 am
https://github.com/fatcerberus/minisphere/blob/master/src/api.c#L496-L521

Looks like the same basic logic to me.  Weird.

I know I had this same problem with miniRT, and I was only able to fix it by centralizing everything to a single script (miniRT.js) which pulls in all the others at the bottom (not the top).  And when I mentally followed the call chain, it made sense--if the RequireScripts are at the top and there's a circular reference, then when you hit the effective "bottom" of the tree (conceptually there is no bottom since it's a circle but in reality there is because of the implicit include guard), Turbo is undefined because all the scripts higher up in the call chain are still in the middle of include resolution, so nobody's actually had a chance to define it yet.

So yeah, not sure why this works in one engine but not the other.

Edit: Shame on you for calling malloc and not checking for null! :P  Actually though, I've stopped checking it myself.  I figure by the time a malloc returns null there's not much I can do in the way of recovery anyway so may as well just let it crash.
Title: Re: minisphere 1.3.1
Post by: Flying Jester on June 19, 2015, 11:54:30 am

Edit: Shame on you for calling malloc and not checking for null! :P  Actually though, I've stopped checking it myself.  I figure by the time a malloc returns null there's not much I can do in the way of recovery anyway so may as well just let it crash.


Yeah, in general there's just no point. In very almost all cases, an OOM is unrecoverable from any perspective. I could fail fast, but if I suspect an OOM I can use a debug malloc that aborts when it would return null.
Title: Re: minisphere 1.3.2
Post by: Fat Cerberus on June 20, 2015, 01:35:48 am
New version up, with a few map engine fixes.

edit: Awesome, 1111 posts! :P
Title: Re: minisphere 1.3.2
Post by: Radnen on June 20, 2015, 02:34:17 am

edit: Awesome, 1111 posts! :P


Don't you mean, 15? :P
Title: Re: minisphere 1.3.2
Post by: Fat Cerberus on June 20, 2015, 02:36:18 am
Why 15?
Title: Re: minisphere 1.3.2
Post by: Fat Cerberus on June 20, 2015, 06:39:12 pm
Tip of the day: Use RNG.name() if you need a unique internal name for something.  The function returns a random string of alphabetic characters, similar to the format of a YouTube video hash:
Code: [Select]
IlysMylJv


The function takes an optional integer parameter specifying the length of the string to generate, up to 255 characters.  If called with no parameter, a default length of 10 is used.
Title: Re: minisphere 1.3.2
Post by: Fat Cerberus on June 21, 2015, 03:05:05 am
https://github.com/fatcerberus/minisphere/commit/313d3dd455d632eb550de578233986c2a45fc5ad

That is all. ;D
Title: Re: minisphere 1.3.2
Post by: Radnen on June 21, 2015, 03:33:39 am
1111 is 15 in binary.

And you have some funny comments in your code. :P
Title: Re: minisphere 1.3.2
Post by: Fat Cerberus on June 21, 2015, 09:30:41 am
Hey, if I have to document the code I might as well have some fun with it, right? :)  You'll find I also tend to write funnier comments when I'm frustrated, like the "HERE BE DRAGONS" ones I have at the top of a few particularly hairy functions.  Then you have my personal favorite minisphere comment of all time, in persons.c:

Code: (c) [Select]
	// check that the person didn't mysteriously disappear...
if (!does_person_exist(person))
return;  // they probably got eaten by a hunger-pig or something.


And of course, this one's a classic too:
Code: (c) [Select]
	// check for obstructing persons
if (!person->ignore_all_persons) {
for (i = 0; i < s_num_persons; ++i) {
if (s_persons[i] == person)  // these persons aren't going to obstruct themselves!
continue;
if (s_persons[i]->layer != layer) continue;  // ignore persons not on the same layer
if (is_person_following(s_persons[i], person)) continue;  // ignore own followers
if (is_person_ignored(person, s_persons[i])) continue;
base = get_person_base(s_persons[i]);
if (do_rects_intersect(my_base, base)) {
is_obstructed = true;
if (out_obstructing_person) *out_obstructing_person = s_persons[i];
break;
}
}
}
Title: Re: minisphere 1.4.0
Post by: Fat Cerberus on June 21, 2015, 11:56:23 am
minisphere 1.4 is up!  I fast-tracked this one since I didn't really have any other ideas for a release, but the CS support is enough of a change to warrant a version bump, methinks.

To be clear: You can now specify .coffee files directly in a RequireScript() call, and require() will recognize .coffee modules as well.  No need to include the CS compiler with your game and manually invoke it, minisphere will load and use it from the system directory automatically.

The funny thing is I don't even really like CoffeeScript that much...

edit: I also recompiled everything with /O2.  It seems I sacrificed a lot of performance when I optimized for size a while back, not really worth it to reduce the engine size by half a MB.
Title: Re: minisphere 1.4.0
Post by: FBnil on June 21, 2015, 05:46:19 pm
Good or bad? Many opinions:
http://code.tutsplus.com/articles/should-you-learn-coffeescript--net-23206

And dipping your toes into CS:
http://blog.teamtreehouse.com/the-absolute-beginners-guide-to-coffeescript
Title: Re: minisphere 1.4.0
Post by: Radnen on June 21, 2015, 06:04:32 pm
Syntactically, coffescript is nice to have for a few areas: Classes, loops, and some areas of logic that may get too verbose. But I've found that libraries like my Link.js go a long ways in making sure iterations on lists and management of larger code sections is easier to maintain.

There's a point in professional JS in which you use code to write less code to code (like my post signature). I find that if you tackle every loop like this:
Code: (javascript) [Select]

for (var i = 0, l = items.length; i < l; ++i) { /**/ }

// rather than:
Link(items).each(function() { /**/ });


Or every class like this:
Code: (javascript) [Select]

var SubClass = function() {
    Class.call(this);
    /* ... */
}

SubClass.prototype = new Class();

// Rather than:
var SubClass = Extend(Class, { /**/ });


Then you are doing JS within JS wrong. CoffeeScript had a good use to write out JS shims for old browsers which isn't exactly needed for Sphere. Sure there are JS features that have been added since Sphere 1.6, but if you use CoffeeScript solely for minisphere I doubt you'll get the most out of it.
Title: Re: minisphere 1.4.0
Post by: Fat Cerberus on June 21, 2015, 06:10:02 pm
Yeah, I personally don't like a lot of things about CS.  Postfix if/unless, for one, confuses me to no end, and indent-based scoping... Ugh.  I prefer to write pure JS.  I still added CS though because I like to give people the option. :). I try not to impose my own preferences on others.
Title: Re: minisphere 1.4.0
Post by: Flying Jester on June 21, 2015, 06:24:36 pm
http://chadaustin.me/2015/05/coffeescript/ (http://chadaustin.me/2015/05/coffeescript/)

I certainly think that the syntax can be just a little ambiguous. I tend to be OK with ws-awareness, it's just a choice, and one I don't mind.

I've only used a little bit of Coffeescript. On the whole, it just seems to be a different syntax for JS (not too much better or worse). The only feature I thought was really nifty was how constructors could put named parameters directly into fields. That would be really nice to have in JS.
Title: Re: minisphere 1.4.0
Post by: FBnil on June 21, 2015, 07:28:06 pm
minimally rewrote agrapher to run with minisphere. It requires the system scripts typeof.js and colors.js (this one is updated with more things).
It draws functions and creates an image of that. See png attached.

So far, although similar at first glance, the grid and lines are offset a little, thus the image is not the same. I'm still investigating what exactly is different.
(s7.png is sphere 1.6 generated and ms.png is minisphere generated).
Title: Re: minisphere 1.4.0
Post by: Fat Cerberus on June 21, 2015, 07:32:26 pm
Just for future reference, you can put system scripts in (game dir)/lib and RequireSystemScript() will find them. :)
Title: Re: minisphere 1.4.0
Post by: FBnil on June 21, 2015, 07:33:39 pm
looking at it with gwenview reveals that the grid (which are transparent blue lines), are not the uniform color they should be, where, a lighter dot should be produced at intersections. This could be because of the way primitives are composed now. While irfanview does not show this difference. (the offset, is viewable in both image viewers, thus a real difference)

The sphere editor image viewer shows even more contrast in the grid produced by minisphere.

@L.E.: Thanks for the tips

edit: I know what it does now: minisphere, when printing a semi transparant color over the black canvas, actually makes the black canvas transparent, instead of blending, then leaving the alpha alone.
Title: Re: minisphere 1.4.0
Post by: Fat Cerberus on June 21, 2015, 07:46:55 pm
I should be able to fix that offsetting easily enough.  I'll take a look at it later.
Title: Re: minisphere 1.4.0
Post by: Fat Cerberus on June 21, 2015, 11:22:51 pm
I fixed the offset.  I just had to add 0.5 to the coordinates passed to Allegro.  Standard off-by-one error.

As for the colors, I think ultimately, with minisphere you're at whim of the hardware when using the legacy primitives.  A screenshot of my result is posted, and as you can see it's actually darker than the image produced by Sphere 1.5.  Which is the exact opposite of the result you got.
Title: Re: minisphere 1.4.1
Post by: Fat Cerberus on June 22, 2015, 12:44:07 am
1.4.1 fixes the off-by-one rendering for Line() as well as a CoffeeScript scoping issue.
Title: Re: minisphere 1.4.1
Post by: FBnil on June 22, 2015, 02:44:34 pm
Ah, it is darker because it is transparent blue, over black, which makes it very black, then rendered as non-transparent pixels (alpha 255).  It seems the transparency of a pixel also affects the RGBA of the destination, while this should only happen when it blends colors.
Title: Re: minisphere 1.4.1
Post by: FBnil on June 22, 2015, 03:56:45 pm
yay! I can segfault minisphere!

1. Run pianola
2. Press pageup twice (changes instrument)
3. press a key, for example q, twice.

Not sure why it doesnt segfault with the other sounds.
Title: Re: minisphere 1.4.1
Post by: Fat Cerberus on June 22, 2015, 05:48:31 pm

Ah, it is darker because it is transparent blue, over black, which makes it very black, then rendered as non-transparent pixels (alpha 255).  It seems the transparency of a pixel also affects the RGBA of the destination, while this should only happen when it blends colors.


Thanks, I'll have to look into that, I may have screwed up the blend modes for surfaces.

Not sure what's causing that segfault though, and I'm not near a computer to run it through the debugger at present.  I'll look at it tonight.
Title: Re: minisphere 1.4.1
Post by: Fat Cerberus on June 22, 2015, 11:49:44 pm
Hm, I can't get it to segfault following your directions.  I did notice however that the pitch doesn't change with different notes, they all sound the same.
Title: Re: minisphere 1.4.1
Post by: Fat Cerberus on June 23, 2015, 12:24:15 am
Well, the monotone notes are now fixed at least.  I forgot to reapply the gain, pitch, etc. when restarting the stream.

Somehow you keep finding the best testcases for minisphere, uncovering so many obscure bugs.  It helps me out a lot. ;)

Now to take another look at that grid.

edit: I figured it out, Allegro by default does blending for all components, including the alpha channel.  I have to use a different function to set a separate blendmode for the alpha channel:
https://www.allegro.cc/manual/al_set_separate_blender

Surprising I didn't notice this before, as I do some blending with surfaces myself.  I'll see about a fix.
Title: Re: minisphere 1.4.1
Post by: Radnen on June 23, 2015, 01:24:42 am

Surprising I didn't notice this before, as I do some blending with surfaces myself.  I'll see about a fix.


Well, full screen consistency can obscure it. I mean, unless you tested with Sphere 1.6 side-by-side you might have not just seen it. ;)
Title: Re: minisphere 1.4.1
Post by: Fat Cerberus on June 23, 2015, 02:07:09 am
@FBnil
The output of your grapher is, for all intents and purposes, identical to Sphere 1.5 now.  See screenshot. :)

The fix will be in 1.4.2.

edit: Dammit, my fix isn't right, I ended up breaking other stuff (like the startup game, and Spectacles).  And just as I was about to release 1.4.2 too... oh well.  Back to the drawing board...
Title: Re: minisphere 1.4.1
Post by: Fat Cerberus on June 23, 2015, 12:28:50 pm
Hm, looks like I'm going to have to leave the alpha blending code the way it is in 1.4.1.  It turns out Sphere 1.6 handles the BLEND mode like this:

Code: (cpp) [Select]
inline void blendRGBA(RGBA& dest, RGBA src)
{
    Blend3(dest, src, src.alpha);
    dest.alpha = 255;
}


Which appears to be impossible to do in hardware--or at least with the blend modes Allegro provides.  In most real-world cases the full RGBA blending works fine, for example the textboxes in Specs and... whatever the startup game does.

For what it's worth, the closest I could come to Sphere's actual behavior is this:
Code: (c) [Select]
al_set_separate_blender(ALLEGRO_ADD, ALLEGRO_ALPHA, ALLEGRO_INVERSE_ALPHA,
ALLEGRO_ADD, ALLEGRO_ZERO, ALLEGRO_ONE);


This leaves the destination's alpha channel alone.  That fixed agrapher, but broke everything else; fully-transparent surfaces stay fully transparent regardless of what you render.  Not particularly useful.
Title: Re: minisphere 1.4.1
Post by: FBnil on June 23, 2015, 03:10:21 pm
Dont break spectacles! fixing the Alpha channel can be done in scripting as well (or maybe a new function). This will become useful for me once spritesets can be updated dynamically (to be able to swap clothes, armour and gear;and if those functions are not available, I can work around that by mixing to disk, and reload the spriteset).
Ill look better into a testcase for the crash on Windows (my platform is Linux now, so maybe it is a library, and not minisphere).

Slowly recovering my ranma demo. You see, long time ago I did remove all duplicate files from my backup, but then lost the original directories (svn) in a laptop crash. Thus, all I have now are spritesets and maps all over the place, without knowing which is newer and more complete.

Lord English, more stuff broke with the current version(Tue Jun 23 02:43:42 2015 -0400), see the attached print demo. All skewed canvas drawings are invisible (or are not being printed). The code is unreadable. I guess I was more focused on functionality at that time. (I guess that is the change that you wanted to roll back).
I also get a bit funky in puff-puff, where NPC's are able to grab powerups. A 9x9 layer is not stretched anymore and surfaces are not mirrored (the firedemo/sun). And in the highscores, the stylewindow is not filled in, something is printed, in very light blue, bot not inside it. Also, the skewing there is weird, the right-side of the logo is not wobbly as expected. I will break that down into smaller bugtests during the week. So do not worry... lots of genuine bugs to come!

There are successes also:
Kamatsu's pathfind.js runs out of the box, as well as my demo around that library (see picture).
I am modifying pathfind.js, as it uses strings as indexes, and numbers are (should be) faster. Will need to benchmark that.

Its fun to read your log. I saw also that stray printf, but thought nothing of it.
Tested the coffee script, but as I do not know it (bar that webpage that had some example code). It was fun to get it in.

mod sound is still a bit off for me.. the channels pan totally right/left, but I'm not sure if that is allegro or that it can be controlled in ms.
Title: Re: minisphere 1.4.1
Post by: Fat Cerberus on June 23, 2015, 03:46:01 pm
Don't know what you mean about the MOD sound, you'll have to explain that a bit more.

I just pushed a commit reverting to the original blending behavior.  Could you test again and see if that fixes the canvas stuff?
Title: Re: minisphere 1.4.1
Post by: FBnil on June 23, 2015, 06:29:02 pm
* print example is 100% ok again (better than 2 build back, as the integration symbol in the formula now displays fine, just as the extra wide plus sign, so something in the resize is now done right again)
* puffpuff highscores still not ok (and the colorbars are back to that because of the old transparency behavior). Image attached. Like I said, will make a simple test for that.
* The sound on Linux seems worse than on windows. But it is in the wine version of Sphere 1.7 too (1.6 with the ctypes working). so a mod is 4 channels: left,right,left,right and not slightly panned off center, as I remember making the music. So it is probably linux. I will have to edit the music in impulse tracker again, create an .it file, and test that. So ignore my comment for the moment, it is not minisphere. (audiere used to sound very good on windows... )
Title: Re: minisphere 1.4.1
Post by: Fat Cerberus on June 23, 2015, 06:38:52 pm
Hm, I wonder what's causing the window to be transparent like that.  As for the blending, the only way I can see to fully emulate Sphere 1.x would be to use a shader.  The fixed-function pipeline just isn't capable of it, and doing surfaces entirely in software is unfeasible as Allegro blitting performance for memory bitmaps is atrocious.
Title: Re: minisphere 1.4.1
Post by: Flying Jester on June 23, 2015, 06:52:54 pm
Does Allegro not allow you to use glBlendFunc and friends?
Title: Re: minisphere 1.4.1
Post by: Fat Cerberus on June 23, 2015, 07:09:22 pm
In theory I could use that, I don't see any combination of parameters which would emulate this however:

RGB = src * alpha + dest * (255 - alpha) / 255
A = 255
Title: Re: minisphere 1.4.1
Post by: Flying Jester on June 23, 2015, 07:41:09 pm
I'm 99% certain you can use the GL FFP to do this.


RGB = src * alpha + dest * (255 - alpha) / 255
A = 255


This would equate to RGB = SRC_RGB * SRC_ALPHA + DEST_RGB * (1 - DEST_ALPHA)?

Perhaps:
Code: (c) [Select]
glBlendColor(0.0f, 0.0f, 0.0f, 1.0f);
glBlendFuncSeparate(GL_SRC_ALPHA, GL_ONE_MINUS_DST_ALPHA, GL_CONSTANT_ALPHA, GL_ZERO);
Title: Re: minisphere 1.4.1
Post by: Fat Cerberus on June 23, 2015, 07:45:08 pm
Indeed, the tricky part is that Sphere does RGB blending using the source alpha, but explicitly sets the alpha of the destination to full opaque.  Kind of odd, but as you can see, a lot of existing code relies on it.

Edit: wait, I see it now.  Never mind me, I need to pay better attention. :)
Title: Re: minisphere 1.4.1
Post by: Fat Cerberus on June 23, 2015, 10:52:41 pm

Code: (c) [Select]
glBlendColor(0.0f, 0.0f, 0.0f, 1.0f);
glBlendFuncSeparate(GL_SRC_ALPHA, GL_ONE_MINUS_DST_ALPHA, GL_CONSTANT_ALPHA, GL_ZERO);



Hm, I did some reading, and glBlendFunc() actually sets the multiplier for the source and destination pixels, even with GL_CONSTANT_COLOR/ALPHA (so the "constant color" is actually a mask and is thus a misnomer).  So basically what your snippet does is overwrite the destination alpha channel with the one from the source image, the same as if you did this:
Code: [Select]
glBlendFuncSeparate(GL_SRC_ALPHA, GL_ONE_MINUS_DST_ALPHA, GL_ONE, GL_ZERO);


It doesn't unconditionally set the destination pixel to full opaque like Sphere does. :(  So yeah, I'm pretty sure I will have to use a shader.  No big deal, I already have the plumbing in place for shaders, it's easy enough to throw a few into the system folder for blending modes.

edit: That said, I did just send a pull request to implement constant-color blending in Allegro, since it's indeed a useful feature and I have no clue how it didn't make it in before now. :P
Title: Re: minisphere 1.4.1
Post by: Fat Cerberus on June 24, 2015, 01:55:55 am
I just implemented engine-wide shader support in minisphere.  Previously shaders only applied to Galileo rendering, and a null shader (i.e. Allegro default) was used everywhere else.  Now minisphere looks for system.vs.glsl and system.fs.glsl in the system directory on startup and uses them if they are found.  These will be used to implement the Sphere 1.x blend and mask modes properly.

The engine will still work if the system shaders are unavailable, in which case it will fall back on the FFP (which will be inaccurate) and the aforementioned null shader.

edit: BAH!  It turns out you can't do blending in a shader because the fragment shader doesn't have access to the destination color, only the source fragment color.  Figures.
Title: Re: minisphere 1.4.1
Post by: Fat Cerberus on June 24, 2015, 01:06:46 pm
@FBnil
Good news, you found an unimplemented feature!  It seems I never implemented gradient windows...

I also got the blending closer to Sphere, but it's still not perfect.

edit: Got it!
Title: Re: minisphere 1.4.1
Post by: FBnil on June 24, 2015, 05:13:34 pm
;D yay! ;D
Title: Re: minisphere 1.4.1
Post by: Fat Cerberus on June 24, 2015, 06:15:37 pm
The only thing I can't figure out is why the fireball on the title isn't displaying.  I added a call to surface.save() to that part, and the image is empty, all black.  No sun.  Yet when I ran through the debugger the color values are right.  Apparently surface.setPixel() is broken.

The other bug I saw with this game is that the death animation isn't right, the car moves to a seemingly random location while it spins out.  I think MapToScreen() is glitched, but it works fine in Specs...
Title: Re: minisphere 1.4.1
Post by: Fat Cerberus on June 25, 2015, 02:53:08 am
Okay, I fixed the fireball/sun thing, and also the mirroring.

I also figured out that MapToScreen() is bugged for small repeating layers due to aliasing (internally it just does a reverse mapping from ScreenToMap, if the layer repeats within a screen it's ambiguous where you are in relation to it).  I'm too tired to come up with a fix now though, I will work on it in the morning.  I'm also not sure why the enemy cars are picking up the power-ups...

edit: MapToScreen*() and ScreenToMap*() are now fixed.
Title: Re: minisphere 1.4.1
Post by: Fat Cerberus on June 25, 2015, 03:14:49 pm
Okay, figured out why cars can pick up flags and stuff in Puff-Puff, it's not technically a bug.  Sphere 1.x only calls the touch-script for input person collisions, minisphere does it for all entities.  Despite that the behavior is not fully compatible, in almost all cases, minisphere's semantics here are preferable.  Persons should be able to respond to anyone touching them, not just the player(s).

For Puff-Puff specifically, the workaround is to add this line to the top of the relevant touch scripts:
Code: (javascript) [Select]
if (GetActivePerson() != GetInputPerson()) return;


So yeah, I think that's all the Puff-Puff bugs, time for 1.4.2!

edit: minisphere 1.4.2 is up.  This fixes an incredible number of bugs, so make sure to check the changelog. ;)
Title: Re: minisphere 1.4.2
Post by: Fat Cerberus on June 26, 2015, 01:24:17 am
Lookie here:
https://travis-ci.org/fatcerberus/minisphere

;D
Title: Re: minisphere 1.4.1
Post by: Radnen on June 26, 2015, 02:42:51 am

Okay, figured out why cars can pick up flags and stuff in Puff-Puff, it's not technically a bug.  Sphere 1.x only calls the touch-script for input person collisions, minisphere does it for all entities.  Despite that the behavior is not fully compatible, in almost all cases, minisphere's semantics here are preferable.  Persons should be able to respond to anyone touching them, not just the player(s).


Despite being incompatible I do like this system. It makes making a RPGMaker like eventing system easier to manage if other entities collisions can be handled gracefully. Otherwise it'd be hard to get back that kind of info without running your own loop of checks (which is rather redundant), and prone to failure.
Title: Re: minisphere 1.4.2
Post by: DaVince on June 26, 2015, 04:36:46 am

Lookie here:
https://travis-ci.org/fatcerberus/minisphere

;D

Nice! I suppose this will warn you of building issues before I get the chance to complain (:P) about it, right? :)
Title: Re: minisphere 1.4.2
Post by: Fat Cerberus on June 26, 2015, 09:04:27 am


Lookie here:
https://travis-ci.org/fatcerberus/minisphere

;D

Nice! I suppose this will warn you of building issues before I get the chance to complain (:P) about it, right? :)


Yes, plus it paves the way for a future official Linux release, as I'll know ahead of time whether the build is broken (so errors don't pile up).  What shocks me is how *silent* the build is, even in the clang build.  No warnings whatsoever.  I wonder what warning level SCons is using...
Title: Re: minisphere 1.4.2
Post by: Flying Jester on June 26, 2015, 11:51:12 am
It looks like the clang build is working in GCC compatibility mode, which is very permissive by Clang's standards.

Try adding CFLAGS=" -Wall " and you will get a barrage of new warnings, and " -pedantic " if you are feeling truly masochistic.
Title: Re: minisphere 1.4.2
Post by: Fat Cerberus on June 26, 2015, 11:58:17 am
I'm just going to go out on a limb here and guess that -pedantic will make the build horribly, horribly broken. :P
Title: Re: minisphere 1.4.2
Post by: Flying Jester on June 26, 2015, 12:21:21 pm
"-pedantic" makes the build 99.9% accurate to the standard you selected (which defaults -std=c99). Clang and GCC are far closer to the C99 standard than MSVC is (or really any other compiler except for Solaris Studio), so I would suspect that it bring up all kinds of new and exciting problems.

You can also add "-ansi" instead of "-std=c99" to get C89, which is very close to what MSVC 2010 has, except that it also disallows single-line comments. C99 didn't really work at all until at least MSVC 2012.
Title: Re: minisphere 1.4.2
Post by: Fat Cerberus on June 26, 2015, 12:23:56 pm
I'm not giving up C99, I like my // comments and sized ints too much. :)
Title: Re: minisphere 1.4.2
Post by: Flying Jester on June 26, 2015, 12:31:58 pm
Well, sized ints are a matter of the library itself, not the standard. While technically a compiler should limit the library a program can see when compiling for a certain standard, this tends to only happen with C++.

Then there's Solaris Studio. (Almost) fully C++14 and C11 compliant, still running on a C++03 and C89 library :/
Title: Re: minisphere 1.4.2
Post by: FBnil on June 26, 2015, 12:50:48 pm
Thanks a TON! It is unbelievable how fast you update and commit.
Good code too, will re-test stuff later-on.
I'm trying to get the networking working, but I seem to either not "getting" it, or there is a bug:

Code: [Select]
  var LocalPort = 9500;
  var Listen = 255; // MAXCONNECT
  var server = new ListeningSocket(LocalPort, Listen);
  if(! server ){Abort("Could not open socket on " + LocalPort)}
  var socket = server.accept();
  if(socket === NULL){Abort("Could not accept socket")}
  if(socket.isConnected()){
    var len = socket.getPendingReadSize();
    if(0 < len && len < 1024){
      var BUFFER = socket.readString(len);
      socket.write("404 Not Found\n\r\n\r");
      Abort(BUFFER);
    }else{
      Abort("len is not ok " + len);
    }
      socket.close();
  }

But for
Code: [Select]
var socket = server.accept();

it throws:
Code: [Select]
JS Error: main.js:10 - TypeError: not callable


I wrapped it in a try loop, of course, but even then, a webbrowser connects, but then "hangs" (as opposed to an immediate "unable to connect" error). So we know it is connecting, but it seems it is not able to start up communications.

Title: Re: minisphere 1.4.2
Post by: FBnil on June 26, 2015, 12:57:12 pm

I'm not giving up C99, I like my // comments and sized ints too much. :)

:D

Wall is allright, and with pedantic... wow...  don't even try it...

Quote
obj/utility.h:6:8: warning: (this will be reported only once per input file)
In file included from obj/animation.c:2:0:
obj/api.h:1:14: warning: ISO C forbids forward references to 'enum' types [-Wpedantic]
typedef enum js_error js_error_t;

(and of course, a waterfall of similar stuff)

Still, some things are worthy of checking out with Wall,
Code: [Select]
CFLAGS = ["-Wall", "-std=c99"]
minisphere = Program("msphere", msphere_sources, LIBS = msphere_libs, CFLAGS = CFLAGS)


Title: Re: minisphere 1.4.2
Post by: Fat Cerberus on June 26, 2015, 01:03:56 pm

Thanks a TON! It is unbelievable how fast you update and commit.
Good code too, will re-test stuff later-on.
I'm trying to get the networking working, but I seem to either not "getting" it, or there is a bug:

Code: [Select]
  var LocalPort = 9500;
  var Listen = 255; // MAXCONNECT
  var server = new ListeningSocket(LocalPort, Listen);
  if(! server ){Abort("Could not open socket on " + LocalPort)}
  var socket = server.accept();
  if(socket === NULL){Abort("Could not accept socket")}
  if(socket.isConnected()){
    var len = socket.getPendingReadSize();
    if(0 < len && len < 1024){
      var BUFFER = socket.readString(len);
      socket.write("404 Not Found\n\r\n\r");
      Abort(BUFFER);
    }else{
      Abort("len is not ok " + len);
    }
      socket.close();
  }

But for
Code: [Select]
var socket = server.accept();

it throws:
Code: [Select]
JS Error: main.js:10 - TypeError: not callable


I wrapped it in a try loop, of course, but even then, a webbrowser connects, but then "hangs" (as opposed to an immediate "unable to connect" error). So we know it is connecting, but it seems it is not able to start up communications.


Oops, I accidentally still had accept() exposed as "acceptNext" from when I originally implemented the Sockets API.  The latest Git build fixes it.
Title: Re: minisphere 1.4.2
Post by: Flying Jester on June 26, 2015, 01:34:05 pm
If -Wall is fine but pedantic is broken, you could also try -Wextra, sort of in between.

...but ideally I suggest at least looking at the output of pedantic.
Title: Re: minisphere 1.4.2
Post by: Fat Cerberus on June 26, 2015, 01:48:36 pm
Amusing side note: Don't ever attempt to use -Wall with MSVC.  It generates thousands of "warnings" that aren't even real warnings--noise like "Oh hey, I inlined this function for you" or "x amount of padding added to this struct" and other such nonsense.
Title: Re: minisphere 1.4.2
Post by: Fat Cerberus on June 26, 2015, 03:13:28 pm
I think I'll throw -pedantic into my SConscript for a bit and see what output I get out of Travis.  I don't have a Linux machine set up yet so I can't test it myself.

edit: Well, I broke the build--std=c99 doesn't define M_PI etc. apparently.

As for pedantic warnings, the only thing I can see is implicit declarations of standard library functions--which means I just forgot an include.  Even Duktape itself was silent.

edit 2: Well that's kind of annoying.  Apparently scons quits after the first file that has errors.  I like MSVC's build system, it will keep going anyway so you get to see if anything else has errors without having to keep restarting the build.
Title: Re: minisphere 1.4.2
Post by: FBnil on June 26, 2015, 05:29:09 pm
MSVC:


See also this: https://www.smore.com/clippy-js
Title: Re: minisphere 1.4.2
Post by: Flying Jester on June 26, 2015, 05:31:05 pm

edit 2: Well that's kind of annoying.  Apparently scons quits after the first file that has errors.  I like MSVC's build system, it will keep going anyway so you get to see if anything else has errors without having to keep restarting the build.


There is some option to change that, but I forget what it is.
Title: Re: minisphere 1.4.2
Post by: Fat Cerberus on June 26, 2015, 06:49:41 pm

MSVC:
[Clippy macro]


Heh... Haha.... ... BWAHAHAHAHAHHAA!  That's perfect.  Absolutely perfect.

I hated that stupid paperclip.  Thank god they had the decency to never implement Agent in MSVC, could you imagine the nightmare? :P

I always did have a soft spot for the cat though.  Annoying as it was I used to enable Agent anyway just to see the cat roll around and play, it was cute.  I turned off all the stupid "hints" though, of course. ;)
Title: Re: minisphere 1.4.2
Post by: Fat Cerberus on June 27, 2015, 01:51:09 am
Hm, I don't think this is too bad:
https://travis-ci.org/fatcerberus/minisphere/jobs/68574196

Rather noisy, but it looks like the vast majority of the warnings are C90 violations (long long, C++-style comments, etc.).  The only ones that really concern me are these:

Code: [Select]
obj/color.c:32:3: warning: initializer element is not computable at load time [enabled by default]


This happens when I declare and initialize a large structure, usually an array of vertices, inline.  The alternative is to set the individual struct members directly, which would be uglier.

So yeah, -pedantic wasn't nearly as scary as I thought it would be.  A testament to how clean the codebase is, I guess. :D
Title: Re: minisphere 1.4.3
Post by: Fat Cerberus on June 27, 2015, 02:59:20 am
minisphere 1.4.3 is up.  This is the first release to be distributed as an installer!  This is convenient, as the installer will automatically associate the engine with SGM and SPK files (the 64-bit engine is associated on 64-bit Windows).  Go try it out!

Oh, I was going to include Sphere Studio, but it doesn't work when run from Program Files, it just terminates immediately, I assume because it tries to save its .ini file in the editor directory...
Title: Re: minisphere 1.4.3
Post by: Fat Cerberus on June 28, 2015, 12:30:34 am
Just updated the 1.4.3 download, the installer now includes Sphere Studio. :D

@Radnen: I hope you don't mind... ;)  Also, check your pull requests. 8)
Title: Re: minisphere 1.4.2
Post by: Flying Jester on June 28, 2015, 12:47:55 am

Rather noisy, but it looks like the vast majority of the warnings are C90 violations (long long, C++-style comments, etc.).  The only ones that really concern me are these:
Code: [Select]
obj/color.c:32:3: warning: initializer element is not computable at load time [enabled by default]



In ansi C (and actually up to C11), initializer lists must be entirely made up of compile-time constants. Of course, common compilers can handle this kind of thing, which is why it ended up being in the C11 standard.

I think you can silence the comment error with "-Wno-comment".
Title: Re: minisphere 1.4.3
Post by: Fat Cerberus on June 28, 2015, 12:46:52 pm
So it seems I broke screenshots at some point... Oops.  I changed it to put them in ~/Sphere Files/Screenshots (home dir, not Sphere's definition of ~/), but forgot to have it create the screenshots folder.  So yeah, screenshots don't work unless you create the folder manually first.  It will be fixed in the next update.
Title: Re: minisphere 1.4.4
Post by: Fat Cerberus on June 28, 2015, 11:52:55 pm
minisphere 1.4.4 is up, fixing the screenshot bug.  The installer now pre-configures Sphere Studio as well (without clobbering existing settings), and creates presets for minisphere.  This should hopefully make the engine much more accessible to newbies! :)

edit: Make that 1.4.5.  I discovered a bug in the installer, it won't give the option to install Sphere Studio.  This fixes it.

I also split minisphere into two downloads.  The GDK includes Sphere Studio and minisphere Console, while the redistributable only includes the standard engine and sample games.
Title: Re: minisphere 1.4.5 with Sphere Studio
Post by: Fat Cerberus on July 02, 2015, 03:10:36 am
So, funny story: I'm all tapped out of ideas for how to improve minisphere any further.  This is probably a good thing since it means the engine is pretty much a complete, comprehensive package, but it's driving me bonkers with boredom. :P

I keep hoping for one more big feature idea so I can get minisphere to v1.5, but no such inspiration yet...

... any ideas? ;)
Title: Re: minisphere 1.4.6 with Sphere Studio
Post by: Fat Cerberus on July 02, 2015, 02:43:03 pm
minisphere 1.4.6 is up with a few minor fixes.  I found a bug where screenshots could be washed out because the backbuffer's alpha channel was incorrectly preserved when saving.  I also fixed the lack of a taskbar icon for games with no icon.png.

Additionally, I dug up the SVG version of the Spherical icon NEO posted a while back and scaled it up 768x768, so now minisphere has a proper Win10-ready icon. :)
Title: Re: minisphere 1.4.6 with Sphere Studio
Post by: Radnen on July 03, 2015, 01:16:41 am

Additionally, I dug up the SVG version of the Spherical icon NEO posted a while back and scaled it up 768x768, so now minisphere has a proper Win10-ready icon. :)


Awesome!
Title: Re: minisphere 1.4.6 with Sphere Studio
Post by: Fat Cerberus on July 04, 2015, 01:55:51 pm
Tip of the day: SphereFS enables creating games which can share save data.  For example, carrying over stats and such to a sequel, or similar such systems.  To do this, use a  ~usr/ path:

~usr/SpectaclesSaga/BrucesStory_fileA.save.

~usr/ points to the same location for all games, so they can share data!
Title: Re: minisphere GDK 1.4.7
Post by: Fat Cerberus on July 05, 2015, 03:01:29 am
New release. :)
Title: Re: minisphere 1.4.7
Post by: N E O on July 05, 2015, 04:26:54 am

Additionally, I dug up the SVG version of the Spherical icon NEO posted a while back and scaled it up 768x768, so now minisphere has a proper Win10-ready icon. :)

It is actually casiotone's, but I lost the original SVG he gave me when we lost the forums so I recreated it.
Title: Re: minisphere 1.4.7
Post by: Fat Cerberus on July 05, 2015, 12:18:18 pm
Did you ever find the gradient version?
Title: Re: minisphere 1.4.7
Post by: N E O on July 05, 2015, 05:49:15 pm
I think the SVG of the gradient version might be on my Windows laptop somewhere, but I haven't had occasion to look for it recently.
Title: commit 8f8e0273652ffe3b39dcc27143dd33c104264a6f
Post by: FBnil on July 08, 2015, 04:55:28 pm
Date:   Wed Jul 8 14:26:40 2015 -0400
Does not run (immediate coredump) on Linux
Title: Re: minisphere 1.4.7
Post by: Fat Cerberus on July 08, 2015, 05:04:16 pm
Hm.  Run with option --log-level 4, what is the output?
Title: Re: minisphere 1.4.7
Post by: FBnil on July 10, 2015, 05:53:29 pm
Sorry, did not read on time. I updated to today and it does not crash anymore. Do you need me to go back in time to compile that version again?
Title: Re: minisphere 1.4.7
Post by: Fat Cerberus on July 10, 2015, 06:11:01 pm
Nah, as long as it's working now that's good.  I probably broke something while I was moving the audio code around.
Title: Re: minisphere 1.4.7
Post by: Fat Cerberus on July 12, 2015, 03:23:13 am
Good news, guys: minisphere 1.5 is only a few days off. :D  Progress has been a bit slow because I'm in the middle of a huge overhaul under the hood of Sphere Studio (I have to earn my collaborator status you know! :P) but I just finished up implementing Mixers and so far everything looks good.  I do still have to add documentation for the Audialis additions and the Mixer object needs a few more features (volume adjustment, for one), but otherwise I'm pretty close to wrapping this up.

Keep in mind that SoundStreams will be limited to 8-bit unsigned in minisphere 1.5.  This makes them easy to manipulate with Sphere ByteArrays, whereas any other sample format would be unwieldy to work with under these circumstances.  Once Duktape releases a version supporting ArrayBuffer/TypedArray, then I'll open up more sample formats. ;)
Title: Re: minisphere 1.5.0
Post by: Fat Cerberus on July 12, 2015, 12:58:58 pm
Okay, 1.5 is up.  Nil, could you try out the new Audialis stuff and let me know how it works?

I removed the demo games from the installers for 1.5.  This got the engine-only down to 2.2mb and the GDK is under 4.5mb.  Not too shabby at all!
Title: Re: minisphere 1.5.1
Post by: Fat Cerberus on July 14, 2015, 12:10:19 pm
I posted minisphere 1.5.1 to fix an issue with creating 8-bit mixers.

To clarify on how the Mixer object works: It actually encapsulates both a hardware voice and a software mixer.  In 1.5.0, both of these were created with the parameters passed to the Mixer constructor.  This failed in the case of 8-bit, as Allegro will create an 8-bit voice but refuses to create an 8-bit mixer.  For good reason, too--it'd be difficult to do any kind of decent mixing at all with only 8 bits of precision! :P

As of 1.5.1, the hardware voice created matches the Mixer() parameters, but the mixer itself is float32.  This ensures the call will succeed so long as the underlying sound hardware supports the requested bit depth and channel count.
Title: Re: minisphere 1.5.1
Post by: Fat Cerberus on July 15, 2015, 03:44:51 pm
So I found a great use for Audialis already: Crossfading BGM.  Previously this would have been a pain, since the BGM volume is generally global, while crossfading requires manipulating individual streams' volume settings.  Under the Sphere 1.x API, something like setting the BGM volume in the middle of a fade would prove to be very tedious indeed.

Under Audialis, I can just set the volume on the BGM mixer when adjusting global volume, and adjust the individual stream volumes separately to handle crossfades.  It makes the code very elegant.
Title: Re: minisphere 1.5.1
Post by: DaVince on July 15, 2015, 05:02:36 pm
I don't really get it. I've successfully done audio1.setVolume(fadeoutvalues); audio2.setVolume(fadeinvalues); before. Or is BGM a separate thing compared to Sphere's standard sound playback functions?
Title: Re: minisphere 1.5.1
Post by: Fat Cerberus on July 15, 2015, 05:12:14 pm
That indeed works, the issue is if someone adjusts the global BGM volume in the midst of a fade, things get tricky because you're in the middle of a tweening operation.
Title: Re: minisphere 1.5.1
Post by: Fat Cerberus on July 15, 2015, 06:24:20 pm
To clarify further, assuming there's a fade in progress:

Code: (javascript) [Select]
mixer.volume = 0.5;


The above won't disturb a crossfade-in-progress.  Further, this allows the global volume to be faded at the same time, for example in a cutscenes where you want to fade down the BGM to half while simultaneously switching tracks.

Another good use would be to implement an in-game volume setting menu.  You create two mixers, one for music and one for sounds, and the volume slider adjusts the volume of the mixers without having to touch the individual Sound objects.

I'm glad I implemented this. :)
Title: Re: minisphere 1.5.1
Post by: N E O on July 16, 2015, 01:19:35 pm
Does 1.5.1 come with an example for using this new Mixer? I also would like updated instructions for Mac compilation (>=Mavericks) so I can try it out, thanks!
Title: Re: minisphere 1.5.1
Post by: Fat Cerberus on July 16, 2015, 01:23:03 pm
No example, but the Mixer and SoundStream objects are included in the API reference (minisphere-api.txt).

As for Mac compilation, it should be the same as Linux. Dependencies are:

* Allegro 5.1.11 (I think available through homebrew?)
* libmng
* zlib

And SCons to build.  Just issue an scons at the command line and you're good to go.  The engine will be built in <checkout dir>/bin/.
Title: Re: minisphere 1.5.2
Post by: Fat Cerberus on July 16, 2015, 02:57:43 pm
@DaVince
If you're still confused about the mixer thing discussed on the last page, check out miniBGM.js in 1.5.2.  You can see how I used the Mixer object to implement crossfading cleanly.
Title: Re: minisphere 1.5.2
Post by: N E O on July 17, 2015, 06:24:14 pm

No example, but the Mixer and SoundStream objects are included in the API reference (minisphere-api.txt).

As for Mac compilation, it should be the same as Linux. Dependencies are:

* Allegro 5.1.11 (I think available through homebrew?)
* libmng
* zlib

And SCons to build.  Just issue an scons at the command line and you're good to go.  The engine will be built in <checkout dir>/bin/.


homebrew doesn't have libmng and it seems to be unobtainable for me at the moment. Is there a command line switch to disable MNG support?
Title: Re: minisphere 1.5.2
Post by: Fat Cerberus on July 17, 2015, 06:28:27 pm
Not at present.  You can probably coax it to compile if you remove animation.c from the SConscript, and then comment out lines 2 and 174 of api.c.
Title: Re: minisphere 1.5.2
Post by: DaVince on July 17, 2015, 06:45:09 pm

@DaVince
If you're still confused about the mixer thing discussed on the last page, check out miniBGM.js in 1.5.2.  You can see how I used the Mixer object to implement crossfading cleanly.

Okey dokes, thanks.
Title: Re: minisphere 1.5.2
Post by: N E O on July 17, 2015, 08:13:42 pm

Not at present.  You can probably coax it to compile if you remove animation.c from the SConscript, and then comment out lines 2 and 174 of api.c.

Two errors:

obj/galileo.c:40:2: error: unknown type name 'ALLEGRO_VERTEX_BUFFER'
obj/galileo.c:393:77: error: use of undeclared identifier 'ALLEGRO_PRIM_BUFFER_STATIC'; did you mean 'ALLEGRO_PRIM_LINE_STRIP'?

Allegro through homebrew is 5.0.11, not 5.1.11 as you recommend. Is this a problem?
Title: Re: minisphere 1.5.2
Post by: Fat Cerberus on July 17, 2015, 08:15:14 pm
You need the latest Allegro (5.1.11).  What version do you have?

Edit:  yeah, 5.0.x won't work.  Not sure what to do here...
Title: Re: minisphere 1.5.2
Post by: Flying Jester on July 17, 2015, 08:37:00 pm
I just had to build it from scratch last I tried on OS X. The build is not especially pleasant on OS X, but it's manageable.
Title: Re: minisphere 1.5.3
Post by: Fat Cerberus on July 18, 2015, 11:17:15 am
1.5.3 adds My Documents\Sphere Games to the search path for GetGameList().  Since minisphere is installed to Program Files, this makes the startup game actually useful again. :P
Title: Re: minisphere 1.5.3
Post by: N E O on July 18, 2015, 03:13:47 pm

I just had to build it from scratch last I tried on OS X. The build is not especially pleasant on OS X, but it's manageable.

This Allegro wiki article (https://wiki.allegro.cc/index.php?title=Install_Allegro5_From_GIT/OSX) says get it from SourceForge, but SF is in "maintenance mode" right now. Don't know how long that will last, so can't build 5.1.11 either.

I don't suppose one of you can put up the Allegro 5.1.11 and libmng source for me to download?
Title: Re: minisphere 1.5.3
Post by: Fat Cerberus on July 18, 2015, 03:28:42 pm
Allegro 5 GitHub mirror:
https://github.com/liballeg/allegro5

Can't speak for libmng though.
Title: Re: minisphere 1.5.3
Post by: N E O on July 18, 2015, 05:09:54 pm
After building Allegro from the GitHub mirror and also removing "mng" from msphere_libs in SConscript and `brew install libjpeg`, new error:

Code: [Select]

Undefined symbols for architecture x86_64:
  "_main", referenced from:
     implicit entry/start for main executable
     (maybe you meant: __al_mangled_main)
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
scons: *** [obj/engine] Error 1
scons: building terminated because of errors.
Title: Re: minisphere 1.5.3
Post by: Fat Cerberus on July 18, 2015, 05:18:25 pm
You have to link to allegro_main on OSX (and only OSX), I forgot about that.  Add it to the SConscript.
Title: Re: minisphere 1.5.3
Post by: N E O on July 18, 2015, 06:59:28 pm

You have to link to allegro_main on OSX (and only OSX), I forgot about that.  Add it to the SConscript.

Just did it, compiles!

Just ran without a startup or any games (it automatically gives an Open File dialog, which is nice), got the following error on the console when canceling:

Code: [Select]

Setting up jump points for longjmp
Looking for a game to launch
Opening game ''
engine(32428,0x10ff94000) malloc: *** error for object 0x7fe271831200: pointer being freed was not allocated
*** set a breakpoint in malloc_error_break to debug
Abort trap: 6


edit: Running a game with just `Abort("Hello, mixer!");` in the game() function results in the following error on closing:

Code: [Select]

Shutting down Audialis
engine(32486,0x104bb1000) malloc: *** error for object 0x7ff8a201fc00: pointer being freed was not allocated
*** set a breakpoint in malloc_error_break to debug
Abort trap: 6
Title: Re: minisphere 1.5.3
Post by: Fat Cerberus on July 18, 2015, 07:04:03 pm
Hm, I'll look into that.  I don't usually cancel the file dialog, so maybe I missed a bug.
Title: Re: minisphere 1.5.3
Post by: N E O on July 18, 2015, 07:16:27 pm
Edited post w/new error.
Title: Re: minisphere 1.5.3
Post by: Fat Cerberus on July 18, 2015, 07:34:19 pm
These are weird.  They look like double free bugs, but I would think they should have shown up in the Windows and Linux builds by now if so.  Does it always crash shutting down Audialis?  Or is it in a different spot each time?
Title: Re: minisphere 1.5.3
Post by: Flying Jester on July 18, 2015, 08:18:45 pm
Try using lldb to see where crash is (or at least the second free)?
Title: Re: minisphere 1.5.3
Post by: N E O on July 18, 2015, 09:26:44 pm

Try using lldb to see where crash is (or at least the second free)?



lldb output for running engine but canceling the Open File:

Code: [Select]

[ ... ]
Setting up jump points for longjmp
Looking for a game to launch
Opening game ''
engine(32663,0x102a9e000) malloc: *** error for object 0x10081be00: pointer being freed was not allocated
*** set a breakpoint in malloc_error_break to debug
Process 32663 stopped
* thread #6: tid = 0x325f34, 0x00007fff96371866 libsystem_kernel.dylib`__pthread_kill + 10, stop reason = signal SIGABRT
    frame #0: 0x00007fff96371866 libsystem_kernel.dylib`__pthread_kill + 10
libsystem_kernel.dylib`__pthread_kill + 10:
-> 0x7fff96371866:  jae    0x7fff96371870            ; __pthread_kill + 20
   0x7fff96371868:  movq   %rax, %rdi
   0x7fff9637186b:  jmp    0x7fff9636e175            ; cerror_nocancel
   0x7fff96371870:  retq  


lldb output for running engine with my one-liner:
Code: [Select]

[ ... ]
Shutting down spriteset manager
Shutting down Audialis
engine(32687,0x102a9e000) malloc: *** error for object 0x101809a00: pointer being freed was not allocated
*** set a breakpoint in malloc_error_break to debug
Process 32687 stopped
* thread #6: tid = 0x3263bb, 0x00007fff96371866 libsystem_kernel.dylib`__pthread_kill + 10, stop reason = signal SIGABRT
    frame #0: 0x00007fff96371866 libsystem_kernel.dylib`__pthread_kill + 10
libsystem_kernel.dylib`__pthread_kill + 10:
-> 0x7fff96371866:  jae    0x7fff96371870            ; __pthread_kill + 20
   0x7fff96371868:  movq   %rax, %rdi
   0x7fff9637186b:  jmp    0x7fff9636e175            ; cerror_nocancel
   0x7fff96371870:  retq  


I don't know how to read debugger, sorry.
Title: Re: minisphere 1.5.3
Post by: Fat Cerberus on July 18, 2015, 09:30:52 pm
No stack trace?

Interesting that it's crashing at the exact same instruction, though.  Which means it's probably the same root cause.  Unfortunately I don't have a Mac to test this on myself.
Title: Re: minisphere 1.5.3
Post by: Flying Jester on July 18, 2015, 09:41:40 pm
At the crash, type `bt` to get a backtrace.

Then, I suggest typing `frame s 1` to see the second frame, and continuing this (but with increasing stack frame numbers) until you get an actual snippet of source with line numbers rather than just assembly.

Really unfortunately, llbd is much more verbose and difficult to use than gdb :(
Title: Re: minisphere 1.5.3
Post by: Fat Cerberus on July 18, 2015, 10:32:58 pm
@NEO:
Okay, from what I can tell from the console output, the first one it looks like the file dialog on OSX is returning true when it shouldn't, judging by this line: Opening game '').  That should be simple enough to fix.

The Audialis shutdown crash I think I have a lead on, but I'd need to see your entire console log, from "Parsing command line" on.
Title: Re: minisphere 1.5.3
Post by: N E O on July 18, 2015, 11:41:20 pm

The Audialis shutdown crash I think I have a lead on, but I'd need to see your entire console log, from "Parsing command line" on.


Dump:
Code: [Select]

Parsing command line
  Game path: <none provided>
  Frameskip limit: 5 frames
  CPU throttle: ON
  Console verbosity: L1

Initializing Allegro
Initializing Dyad
Loading system configuration
Initializing async manager
Initializing Mersenne Twister
Initializing Galileo
Initializing Audialis
Initializing input
Initializing spriteset manager
Initializing map engine
Initializing persons manager
Initializing Duktape
Initializing CoffeeScript
  Error evaluating compiler script  RangeError: compiler recursion limit (line 8)'CoffeeScript' not definedInitializing Sphere API
  v2.0 (minisphere 1.5.2)
  minisphere
  sphere-legacy-api
  sphere-obj-constructors
  sphere-obj-props
  sphere-audialis
  sphere-coffeescript
  sphere-commonjs
  sphere-galileo
  sphere-map-engine
  sphere-spherefs
  minisphere-async-api
  minisphere-new-sockets
  minisphere-rng-object
  minisphere-galileo-shaders
  frameskip-api
  set-script-function
Setting up jump points for longjmp
Looking for a game to launch
Opening game '/Users/lux/dev/minisphere/bin/games/nmix/game.sgm'
  Title: NMix
  Author: NeoLogiX
  Resolution: 320x240
Creating render window
Initializing shader manager

Fragment shader compile log:
ERROR: 0:10: 'blend_mode' : syntax error syntax error

  System shaders not found
Loading system font
Calling game()

JS Error: game.js:6 - Error: Hello, mixer!
Shutting down map engine
Shutting down persons manager
Shutting down input
Shutting down Duktape
Shutting down Dyad
Shutting down spriteset manager
Shutting down Audialis
engine(32687,0x102a9e000) malloc: *** error for object 0x101809a00: pointer being freed was not allocated
*** set a breakpoint in malloc_error_break to debug
Process 32687 stopped
* thread #6: tid = 0x3263bb, 0x00007fff96371866 libsystem_kernel.dylib`__pthread_kill + 10, stop reason = signal SIGABRT
    frame #0: 0x00007fff96371866 libsystem_kernel.dylib`__pthread_kill + 10
libsystem_kernel.dylib`__pthread_kill + 10:
-> 0x7fff96371866:  jae    0x7fff96371870            ; __pthread_kill + 20
   0x7fff96371868:  movq   %rax, %rdi
   0x7fff9637186b:  jmp    0x7fff9636e175            ; cerror_nocancel
   0x7fff96371870:  retq  


Re backtrace - `bt` and interesting frames are as follows:

Code: [Select]

* thread #6: tid = 0x33298d, 0x00007fff96371866 libsystem_kernel.dylib`__pthread_kill + 10, stop reason = signal SIGABRT
  * frame #0: 0x00007fff96371866 libsystem_kernel.dylib`__pthread_kill + 10
    frame #1: 0x00007fff92a1c35c libsystem_pthread.dylib`pthread_kill + 92
    frame #2: 0x00007fff96f58b1a libsystem_c.dylib`abort + 125
    frame #3: 0x00007fff9161707f libsystem_malloc.dylib`free + 411
    frame #4: 0x00000001000e29c6 liballegro_audio.5.1.dylib`_aqueue_deallocate_voice(voice=0x00000001005359f0) + 70 at aqueue.m:212
    frame #5: 0x00000001000e1a38 liballegro_audio.5.1.dylib`al_destroy_voice(voice=0x00000001005359f0) + 40 at kcm_voice.c:114
    frame #6: 0x0000000100164bab liballegro.5.1.dylib`_al_run_destructors(dtors=0x0000000100535980) + 155 at dtor.c:117
    frame #7: 0x00000001000d8675 liballegro_audio.5.1.dylib`_al_kcm_shutdown_destructors + 21 at kcm_dtor.c:46
    frame #8: 0x00000001000d74e8 liballegro_audio.5.1.dylib`al_uninstall_audio + 24 at audio.c:401
    frame #9: 0x0000000100166108 liballegro.5.1.dylib`_al_run_exit_funcs + 136 at exitfunc.c:92
    frame #10: 0x000000010016cac5 liballegro.5.1.dylib`al_uninstall_system + 21 at system.c:298
    frame #11: 0x00007fff96f59794 libsystem_c.dylib`__cxa_finalize + 164
    frame #12: 0x00007fff96f59a4c libsystem_c.dylib`exit + 22
    frame #13: 0x00000001001a77be liballegro.5.1.dylib`+[AllegroAppDelegate app_main:] [inlined] call_user_main + 30 at osx_app_delegate.m:214
    frame #14: 0x00000001001a77a4 liballegro.5.1.dylib`+[AllegroAppDelegate app_main:](self=<unavailable>, _cmd=<unavailable>, arg=<unavailable>) + 4 at osx_app_delegate.m:225
    frame #15: 0x00007fff917d0d8b Foundation`__NSThread__main__ + 1318
    frame #16: 0x00007fff92a1b899 libsystem_pthread.dylib`_pthread_body + 138
    frame #17: 0x00007fff92a1b72a libsystem_pthread.dylib`_pthread_start + 137
    frame #18: 0x00007fff92a1ffc9 libsystem_pthread.dylib`thread_start + 13


Code: [Select]

frame #4: 0x00000001000e29c6 liballegro_audio.5.1.dylib`_aqueue_deallocate_voice(voice=0x00000001005359f0) + 70 at aqueue.m:212
   209 static void _aqueue_deallocate_voice(ALLEGRO_VOICE *voice)
   210 {
   211    al_free(voice->extra);
-> 212    al_free(silence);
   213    voice->extra = NULL;
   214 }
   215

frame #5: 0x00000001000e1a38 liballegro_audio.5.1.dylib`al_destroy_voice(voice=0x00000001005359f0) + 40 at kcm_voice.c:114
   111       ASSERT(al_get_voice_playing(voice) == false);
   112
   113       /* We do NOT lock the voice mutex when calling this method. */
-> 114       voice->driver->deallocate_voice(voice);
   115       al_destroy_mutex(voice->mutex);
   116       al_destroy_cond(voice->cond);
   117



The problem very well could be audio. Among other things, I got warnings about missing libdumb and libvorbis for some reason during early attempts to compile, and when I followed the wrong instructions early on and CMake'ed it to be a Framework it kept giving "unexpectedly closed" errors during the process.
Title: Re: minisphere 1.5.3
Post by: Fat Cerberus on July 19, 2015, 12:04:40 am
Looks like the crash is inside of Allegro.  What's interesting is that it's crashing on trying to free the silence buffer...  Looks like I'll be putting in another bug report. :P

Also odd: CoffeeScript errors out, and the shaders fail to compile also.  Not sure what's going on there!
Title: Re: minisphere 1.5.3
Post by: Fat Cerberus on July 19, 2015, 03:02:42 am
Interesting, apparently for whatever reason your system doesn't like uint uniforms in shaders.  That same shader compiles fine on every Windows machine I've tested on, although I can't speak for Linux as I've never seen DaVince's startup listing.  In any case, the uniform in question wasn't even used (it was part of a failed attempt to implement blending in the fragment shader), so I removed it and also cleaned up the CoffeeScript output (I forgot to add newlines).

I did manage to fix (or think I fixed) the crash on canceling the file dialog.  Apparently al_show_native_file_dialog() returns true on OS X on cancel, even though it returns false in that circumstance seemingly everywhere else.  Then again, I probably just misinterpreted the documentation:
Code: [Select]
Returns true on success, false on failure.


I originally interpreted "failure" to mean "the user closed the dialog", but that might not have been the intent, since Allegro provides a different function, al_get_native_file_dialog_count(), which returns the number of files selected.  If the user cancels, the documentation says explicitly that this will return zero.  So that's what I used.
Title: Re: minisphere 1.5.3
Post by: N E O on July 19, 2015, 01:46:59 pm
*** set a breakpoint in malloc_error_break to debug

Just compiled 1.5.3 and same errors when trying to free audio, but seems like it gets there much faster now so whatever you did to fix OSX open file dialog seems to work. How do I set a breakpoint to confirm the above?
Title: Re: minisphere 1.5.3
Post by: Fat Cerberus on July 19, 2015, 11:47:50 pm
I don't know anything about lldb, but could you checkout the latest source, run the engine with the option --log-level 4 and post the console output?
Title: Re: minisphere 1.5.3
Post by: Flying Jester on July 20, 2015, 01:03:33 am
I don't think setting a breakpoint there would help much unless you had a really good knowledge of Allegro. You basically need to know exactly what is going on in the function directly above the one you set a breakpoint at the edge of.

In any case, in lldb it looks a little like this:
Code: [Select]
breakpoint set -f <filename> -l <line>

You need to set breakpoints either before you type `run` or at the prompt you get by hitting ctrl+c at the terminal while the program is running (or when another breakpoint is triggered, either from a set breakpoint or from a segfault)

I don't recall the syntax for setting it at edges of certain functions. This is where lldb's syntax is really annoying. Everywhere else it's basically just an extra letter here or there, but with breakpoints its fairly obtuse compared to gdb, Intel's debugger, and MS's command line debugger.
Title: Re: minisphere 1.5.3
Post by: Fat Cerberus on July 20, 2015, 01:45:54 am
MS has a command-line debugger?  I never knew that.  The only one I ever remember is the DEBUG command in DOS.  And that was more of a disassembler than a debugger anyway.

As for this crash, the most interesting part to me is the stack trace.  There is no user code on the stack at all--only Allegro functions and system calls.  this makes no sense to me since the last thing printed to the console is "Shutting down Audialis", whereas in a clean shutdown there are a few more steps after that.  So there should definitely be engine code on the stack at that point, but there clearly isn't.  I'm baffled.

edit: Hey, I have an idea!  Any chance you could run the engine under valgrind?  That tends to find weird memory bugs like this.
Title: Re: minisphere 1.5.3
Post by: N E O on July 20, 2015, 02:53:35 pm

I don't know anything about lldb, but could you checkout the latest source, run the engine with the option --log-level 4 and post the console output?


Log for canceling game load:
Code: [Select]

Parsing command line
  Game path: <none provided>
  Frameskip limit: 5 frames
  CPU throttle: ON
  Console verbosity: L4

Initializing Allegro
Initializing Dyad
Loading system configuration
Opening File 0 as '~sys/system.ini'
Initializing async manager
Initializing Mersenne Twister
Seeding MT19937 (1437418155)
Initializing Galileo
Initializing Audialis
Creating new Mixer 0 at 44 kHz
  Format: 2ch 44100 Hz, 16-bit
Incrementing Mixer 0 refcount, new: 1
Initializing input
Opening File 1 as '~sys/../minisphere.conf'
Reading key 'keymap_Player1_MENU' from File 1
Reading key 'keymap_Player1_UP' from File 1
Reading key 'keymap_Player1_DOWN' from File 1
Reading key 'keymap_Player1_LEFT' from File 1
Reading key 'keymap_Player1_RIGHT' from File 1
Reading key 'keymap_Player1_A' from File 1
Reading key 'keymap_Player1_B' from File 1
Reading key 'keymap_Player1_X' from File 1
Reading key 'keymap_Player1_Y' from File 1
Reading key 'keymap_Player2_MENU' from File 1
Reading key 'keymap_Player2_UP' from File 1
Reading key 'keymap_Player2_DOWN' from File 1
Reading key 'keymap_Player2_LEFT' from File 1
Reading key 'keymap_Player2_RIGHT' from File 1
Reading key 'keymap_Player2_A' from File 1
Reading key 'keymap_Player2_B' from File 1
Reading key 'keymap_Player2_X' from File 1
Reading key 'keymap_Player2_Y' from File 1
Reading key 'keymap_Player3_MENU' from File 1
Reading key 'keymap_Player3_UP' from File 1
Reading key 'keymap_Player3_DOWN' from File 1
Reading key 'keymap_Player3_LEFT' from File 1
Reading key 'keymap_Player3_RIGHT' from File 1
Reading key 'keymap_Player3_A' from File 1
Reading key 'keymap_Player3_B' from File 1
Reading key 'keymap_Player3_X' from File 1
Reading key 'keymap_Player3_Y' from File 1
Reading key 'keymap_Player4_MENU' from File 1
Reading key 'keymap_Player4_UP' from File 1
Reading key 'keymap_Player4_DOWN' from File 1
Reading key 'keymap_Player4_LEFT' from File 1
Reading key 'keymap_Player4_RIGHT' from File 1
Reading key 'keymap_Player4_A' from File 1
Reading key 'keymap_Player4_B' from File 1
Reading key 'keymap_Player4_X' from File 1
Reading key 'keymap_Player4_Y' from File 1
File 1 no longer in use, deallocating
Initializing spriteset manager
Initializing map engine
Initializing persons manager
Initializing JavaScript
  Duktape v1.2.2
  Error evaluating compiler script
  RangeError: compiler recursion limit (line 8)
  CoffeeScript support not enabled
Initializing Sphere API
  v2.0 (minisphere 1.5.3)
  minisphere
  sphere-legacy-api
  sphere-obj-constructors
  sphere-obj-props
  sphere-audialis
  sphere-coffeescript
  sphere-commonjs
  sphere-galileo
  sphere-map-engine
  sphere-spherefs
  minisphere-async-api
  minisphere-galileo-shaders
  minisphere-new-sockets
  minisphere-rng-object
  frameskip-api
  set-script-function
Reading key 'Arrow' from File 0
Loading Image 0 as '~sys/pointer.png'
Reading key 'UpArrow' from File 0
Loading Image 1 as '~sys/up_arrow.png'
Reading key 'DownArrow' from File 0
Loading Image 2 as '~sys/down_arrow.png'
Reading key 'WindowStyle' from File 0
Reading 16x16 Image 3 from open file
Reading 16x16 Image 4 from open file
Reading 16x16 Image 5 from open file
Reading 16x16 Image 6 from open file
Reading 16x16 Image 7 from open file
Reading 16x16 Image 8 from open file
Reading 16x16 Image 9 from open file
Reading 16x16 Image 10 from open file
Reading 16x16 Image 11 from open file
Setting up jump points for longjmp
Looking for a game to launch
engine(37145,0x10f111000) malloc: *** error for object 0x7fd01480da00: pointer being freed was not allocated
*** set a breakpoint in malloc_error_break to debug
Abort trap: 6


Re valgrind - I don't know how to use valgrind, but homebrew seems to let me install it.
Title: Re: minisphere 1.5.3
Post by: Fat Cerberus on July 20, 2015, 03:40:59 pm
And for your little demo game?
Title: Re: minisphere 1.5.3
Post by: N E O on July 20, 2015, 05:11:30 pm

And for your little demo game?


Sorry, I forgot, here it is:

Code: [Select]

Parsing command line
  Game path: <none provided>
  Frameskip limit: 5 frames
  CPU throttle: ON
  Console verbosity: L4

Initializing Allegro
Initializing Dyad
Loading system configuration
Opening File 0 as '~sys/system.ini'
Initializing async manager
Initializing Mersenne Twister
Seeding MT19937 (1437426614)
Initializing Galileo
Initializing Audialis
Creating new Mixer 0 at 44 kHz
  Format: 2ch 44100 Hz, 16-bit
Incrementing Mixer 0 refcount, new: 1
Initializing input
Opening File 1 as '~sys/../minisphere.conf'
Reading key 'keymap_Player1_MENU' from File 1
Reading key 'keymap_Player1_UP' from File 1
Reading key 'keymap_Player1_DOWN' from File 1
Reading key 'keymap_Player1_LEFT' from File 1
Reading key 'keymap_Player1_RIGHT' from File 1
Reading key 'keymap_Player1_A' from File 1
Reading key 'keymap_Player1_B' from File 1
Reading key 'keymap_Player1_X' from File 1
Reading key 'keymap_Player1_Y' from File 1
Reading key 'keymap_Player2_MENU' from File 1
Reading key 'keymap_Player2_UP' from File 1
Reading key 'keymap_Player2_DOWN' from File 1
Reading key 'keymap_Player2_LEFT' from File 1
Reading key 'keymap_Player2_RIGHT' from File 1
Reading key 'keymap_Player2_A' from File 1
Reading key 'keymap_Player2_B' from File 1
Reading key 'keymap_Player2_X' from File 1
Reading key 'keymap_Player2_Y' from File 1
Reading key 'keymap_Player3_MENU' from File 1
Reading key 'keymap_Player3_UP' from File 1
Reading key 'keymap_Player3_DOWN' from File 1
Reading key 'keymap_Player3_LEFT' from File 1
Reading key 'keymap_Player3_RIGHT' from File 1
Reading key 'keymap_Player3_A' from File 1
Reading key 'keymap_Player3_B' from File 1
Reading key 'keymap_Player3_X' from File 1
Reading key 'keymap_Player3_Y' from File 1
Reading key 'keymap_Player4_MENU' from File 1
Reading key 'keymap_Player4_UP' from File 1
Reading key 'keymap_Player4_DOWN' from File 1
Reading key 'keymap_Player4_LEFT' from File 1
Reading key 'keymap_Player4_RIGHT' from File 1
Reading key 'keymap_Player4_A' from File 1
Reading key 'keymap_Player4_B' from File 1
Reading key 'keymap_Player4_X' from File 1
Reading key 'keymap_Player4_Y' from File 1
File 1 no longer in use, deallocating
Initializing spriteset manager
Initializing map engine
Initializing persons manager
Initializing JavaScript
  Duktape v1.2.2
  Error evaluating compiler script
  RangeError: compiler recursion limit (line 8)
  CoffeeScript support not enabled
Initializing Sphere API
  v2.0 (minisphere 1.5.3)
  minisphere
  sphere-legacy-api
  sphere-obj-constructors
  sphere-obj-props
  sphere-audialis
  sphere-coffeescript
  sphere-commonjs
  sphere-galileo
  sphere-map-engine
  sphere-spherefs
  minisphere-async-api
  minisphere-galileo-shaders
  minisphere-new-sockets
  minisphere-rng-object
  frameskip-api
  set-script-function
Reading key 'Arrow' from File 0
Loading Image 0 as '~sys/pointer.png'
Reading key 'UpArrow' from File 0
Loading Image 1 as '~sys/up_arrow.png'
Reading key 'DownArrow' from File 0
Loading Image 2 as '~sys/down_arrow.png'
Reading key 'WindowStyle' from File 0
Reading 16x16 Image 3 from open file
Reading 16x16 Image 4 from open file
Reading 16x16 Image 5 from open file
Reading 16x16 Image 6 from open file
Reading 16x16 Image 7 from open file
Reading 16x16 Image 8 from open file
Reading 16x16 Image 9 from open file
Reading 16x16 Image 10 from open file
Reading 16x16 Image 11 from open file
Setting up jump points for longjmp
Looking for a game to launch
Opening game '/Users/lux/dev/minisphere/bin/games/nmix/game.sgm'
  Title: NMix
  Author: NeoLogiX
  Resolution: 320x240
Creating render window
Loading Image 12 as '~sgm/icon.png'
  Failed to load Image 12Loading Image 12 as '~sys/icon.png'
Decrementing Image 12 refcount, new: 0
Image 12 no longer in use, deallocating
Opening File 2 as 'keymap.mini'
Reading key 'keymap_Player1_MENU' from File 2
Reading key 'keymap_Player1_UP' from File 2
Reading key 'keymap_Player1_DOWN' from File 2
Reading key 'keymap_Player1_LEFT' from File 2
Reading key 'keymap_Player1_RIGHT' from File 2
Reading key 'keymap_Player1_A' from File 2
Reading key 'keymap_Player1_B' from File 2
Reading key 'keymap_Player1_X' from File 2
Reading key 'keymap_Player1_Y' from File 2
Reading key 'keymap_Player2_MENU' from File 2
Reading key 'keymap_Player2_UP' from File 2
Reading key 'keymap_Player2_DOWN' from File 2
Reading key 'keymap_Player2_LEFT' from File 2
Reading key 'keymap_Player2_RIGHT' from File 2
Reading key 'keymap_Player2_A' from File 2
Reading key 'keymap_Player2_B' from File 2
Reading key 'keymap_Player2_X' from File 2
Reading key 'keymap_Player2_Y' from File 2
Reading key 'keymap_Player3_MENU' from File 2
Reading key 'keymap_Player3_UP' from File 2
Reading key 'keymap_Player3_DOWN' from File 2
Reading key 'keymap_Player3_LEFT' from File 2
Reading key 'keymap_Player3_RIGHT' from File 2
Reading key 'keymap_Player3_A' from File 2
Reading key 'keymap_Player3_B' from File 2
Reading key 'keymap_Player3_X' from File 2
Reading key 'keymap_Player3_Y' from File 2
Reading key 'keymap_Player4_MENU' from File 2
Reading key 'keymap_Player4_UP' from File 2
Reading key 'keymap_Player4_DOWN' from File 2
Reading key 'keymap_Player4_LEFT' from File 2
Reading key 'keymap_Player4_RIGHT' from File 2
Reading key 'keymap_Player4_A' from File 2
Reading key 'keymap_Player4_B' from File 2
Reading key 'keymap_Player4_X' from File 2
Reading key 'keymap_Player4_Y' from File 2
File 2 no longer in use, deallocating
Initializing shader support
Loading system font
Reading key 'Font' from File 0
Loading Font 0 as '~sys/system.rfn'
Creating Image 13 at 128x192
Creating Image 14 as 8x12 subimage of Image 13
Creating Image 15 as 8x12 subimage of Image 13
Creating Image 16 as 8x12 subimage of Image 13
Creating Image 17 as 8x12 subimage of Image 13
Creating Image 18 as 8x12 subimage of Image 13
Creating Image 19 as 8x12 subimage of Image 13
Creating Image 20 as 8x12 subimage of Image 13
Creating Image 21 as 8x12 subimage of Image 13
Creating Image 22 as 8x12 subimage of Image 13
Creating Image 23 as 8x12 subimage of Image 13
Creating Image 24 as 8x12 subimage of Image 13
Creating Image 25 as 8x12 subimage of Image 13
Creating Image 26 as 8x12 subimage of Image 13
Creating Image 27 as 8x12 subimage of Image 13
Creating Image 28 as 8x12 subimage of Image 13
Creating Image 29 as 8x12 subimage of Image 13
Creating Image 30 as 8x12 subimage of Image 13
Creating Image 31 as 8x12 subimage of Image 13
Creating Image 32 as 8x12 subimage of Image 13
Creating Image 33 as 8x12 subimage of Image 13
Creating Image 34 as 8x12 subimage of Image 13
Creating Image 35 as 8x12 subimage of Image 13
Creating Image 36 as 8x12 subimage of Image 13
Creating Image 37 as 8x12 subimage of Image 13
Creating Image 38 as 8x12 subimage of Image 13
Creating Image 39 as 8x12 subimage of Image 13
Creating Image 40 as 8x12 subimage of Image 13
Creating Image 41 as 8x12 subimage of Image 13
Creating Image 42 as 8x12 subimage of Image 13
Creating Image 43 as 8x12 subimage of Image 13
Creating Image 44 as 8x12 subimage of Image 13
Creating Image 45 as 8x12 subimage of Image 13
Creating Image 46 as 8x12 subimage of Image 13
Creating Image 47 as 3x12 subimage of Image 13
Creating Image 48 as 5x12 subimage of Image 13
Creating Image 49 as 8x12 subimage of Image 13
Creating Image 50 as 7x12 subimage of Image 13
Creating Image 51 as 8x12 subimage of Image 13
Creating Image 52 as 8x12 subimage of Image 13
Creating Image 53 as 3x12 subimage of Image 13
Creating Image 54 as 5x12 subimage of Image 13
Creating Image 55 as 5x12 subimage of Image 13
Creating Image 56 as 7x12 subimage of Image 13
Creating Image 57 as 7x12 subimage of Image 13
Creating Image 58 as 4x12 subimage of Image 13
Creating Image 59 as 5x12 subimage of Image 13
Creating Image 60 as 3x12 subimage of Image 13
Creating Image 61 as 6x12 subimage of Image 13
Creating Image 62 as 7x12 subimage of Image 13
Creating Image 63 as 7x12 subimage of Image 13
Creating Image 64 as 6x12 subimage of Image 13
Creating Image 65 as 6x12 subimage of Image 13
Creating Image 66 as 7x12 subimage of Image 13
Creating Image 67 as 6x12 subimage of Image 13
Creating Image 68 as 7x12 subimage of Image 13
Creating Image 69 as 7x12 subimage of Image 13
Creating Image 70 as 7x12 subimage of Image 13
Creating Image 71 as 7x12 subimage of Image 13
Creating Image 72 as 3x12 subimage of Image 13
Creating Image 73 as 4x12 subimage of Image 13
Creating Image 74 as 6x12 subimage of Image 13
Creating Image 75 as 7x12 subimage of Image 13
Creating Image 76 as 6x12 subimage of Image 13
Creating Image 77 as 6x12 subimage of Image 13
Creating Image 78 as 8x12 subimage of Image 13
Creating Image 79 as 7x12 subimage of Image 13
Creating Image 80 as 7x12 subimage of Image 13
Creating Image 81 as 7x12 subimage of Image 13
Creating Image 82 as 7x12 subimage of Image 13
Creating Image 83 as 7x12 subimage of Image 13
Creating Image 84 as 7x12 subimage of Image 13
Creating Image 85 as 7x12 subimage of Image 13
Creating Image 86 as 7x12 subimage of Image 13
Creating Image 87 as 5x12 subimage of Image 13
Creating Image 88 as 6x12 subimage of Image 13
Creating Image 89 as 8x12 subimage of Image 13
Creating Image 90 as 7x12 subimage of Image 13
Creating Image 91 as 7x12 subimage of Image 13
Creating Image 92 as 7x12 subimage of Image 13
Creating Image 93 as 7x12 subimage of Image 13
Creating Image 94 as 7x12 subimage of Image 13
Creating Image 95 as 7x12 subimage of Image 13
Creating Image 96 as 8x12 subimage of Image 13
Creating Image 97 as 6x12 subimage of Image 13
Creating Image 98 as 7x12 subimage of Image 13
Creating Image 99 as 7x12 subimage of Image 13
Creating Image 100 as 7x12 subimage of Image 13
Creating Image 101 as 7x12 subimage of Image 13
Creating Image 102 as 7x12 subimage of Image 13
Creating Image 103 as 7x12 subimage of Image 13
Creating Image 104 as 7x12 subimage of Image 13
Creating Image 105 as 4x12 subimage of Image 13
Creating Image 106 as 6x12 subimage of Image 13
Creating Image 107 as 4x12 subimage of Image 13
Creating Image 108 as 7x12 subimage of Image 13
Creating Image 109 as 8x12 subimage of Image 13
Creating Image 110 as 4x12 subimage of Image 13
Creating Image 111 as 7x12 subimage of Image 13
Creating Image 112 as 7x12 subimage of Image 13
Creating Image 113 as 7x12 subimage of Image 13
Creating Image 114 as 7x12 subimage of Image 13
Creating Image 115 as 7x12 subimage of Image 13
Creating Image 116 as 6x12 subimage of Image 13
Creating Image 117 as 7x12 subimage of Image 13
Creating Image 118 as 7x12 subimage of Image 13
Creating Image 119 as 5x12 subimage of Image 13
Creating Image 120 as 5x12 subimage of Image 13
Creating Image 121 as 7x12 subimage of Image 13
Creating Image 122 as 5x12 subimage of Image 13
Creating Image 123 as 7x12 subimage of Image 13
Creating Image 124 as 7x12 subimage of Image 13
Creating Image 125 as 7x12 subimage of Image 13
Creating Image 126 as 7x12 subimage of Image 13
Creating Image 127 as 7x12 subimage of Image 13
Creating Image 128 as 6x12 subimage of Image 13
Creating Image 129 as 7x12 subimage of Image 13
Creating Image 130 as 6x12 subimage of Image 13
Creating Image 131 as 7x12 subimage of Image 13
Creating Image 132 as 7x12 subimage of Image 13
Creating Image 133 as 7x12 subimage of Image 13
Creating Image 134 as 7x12 subimage of Image 13
Creating Image 135 as 7x12 subimage of Image 13
Creating Image 136 as 7x12 subimage of Image 13
Creating Image 137 as 5x12 subimage of Image 13
Creating Image 138 as 3x12 subimage of Image 13
Creating Image 139 as 5x12 subimage of Image 13
Creating Image 140 as 8x12 subimage of Image 13
Creating Image 141 as 6x12 subimage of Image 13
Creating Image 142 as 8x12 subimage of Image 13
Creating Image 143 as 8x12 subimage of Image 13
Creating Image 144 as 8x12 subimage of Image 13
Creating Image 145 as 8x12 subimage of Image 13
Creating Image 146 as 8x12 subimage of Image 13
Creating Image 147 as 8x12 subimage of Image 13
Creating Image 148 as 8x12 subimage of Image 13
Creating Image 149 as 8x12 subimage of Image 13
Creating Image 150 as 8x12 subimage of Image 13
Creating Image 151 as 8x12 subimage of Image 13
Creating Image 152 as 8x12 subimage of Image 13
Creating Image 153 as 8x12 subimage of Image 13
Creating Image 154 as 8x12 subimage of Image 13
Creating Image 155 as 8x12 subimage of Image 13
Creating Image 156 as 8x12 subimage of Image 13
Creating Image 157 as 8x12 subimage of Image 13
Creating Image 158 as 8x12 subimage of Image 13
Creating Image 159 as 8x12 subimage of Image 13
Creating Image 160 as 8x12 subimage of Image 13
Creating Image 161 as 8x12 subimage of Image 13
Creating Image 162 as 8x12 subimage of Image 13
Creating Image 163 as 8x12 subimage of Image 13
Creating Image 164 as 8x12 subimage of Image 13
Creating Image 165 as 8x12 subimage of Image 13
Creating Image 166 as 8x12 subimage of Image 13
Creating Image 167 as 8x12 subimage of Image 13
Creating Image 168 as 8x12 subimage of Image 13
Creating Image 169 as 8x12 subimage of Image 13
Creating Image 170 as 8x12 subimage of Image 13
Creating Image 171 as 8x12 subimage of Image 13
Creating Image 172 as 8x12 subimage of Image 13
Creating Image 173 as 8x12 subimage of Image 13
Creating Image 174 as 8x12 subimage of Image 13
Creating Image 175 as 8x12 subimage of Image 13
Creating Image 176 as 8x12 subimage of Image 13
Creating Image 177 as 8x12 subimage of Image 13
Creating Image 178 as 8x12 subimage of Image 13
Creating Image 179 as 8x12 subimage of Image 13
Creating Image 180 as 8x12 subimage of Image 13
Creating Image 181 as 8x12 subimage of Image 13
Creating Image 182 as 8x12 subimage of Image 13
Creating Image 183 as 8x12 subimage of Image 13
Creating Image 184 as 8x12 subimage of Image 13
Creating Image 185 as 8x12 subimage of Image 13
Creating Image 186 as 8x12 subimage of Image 13
Creating Image 187 as 8x12 subimage of Image 13
Creating Image 188 as 8x12 subimage of Image 13
Creating Image 189 as 8x12 subimage of Image 13
Creating Image 190 as 8x12 subimage of Image 13
Creating Image 191 as 8x12 subimage of Image 13
Creating Image 192 as 8x12 subimage of Image 13
Creating Image 193 as 8x12 subimage of Image 13
Creating Image 194 as 8x12 subimage of Image 13
Creating Image 195 as 8x12 subimage of Image 13
Creating Image 196 as 8x12 subimage of Image 13
Creating Image 197 as 8x12 subimage of Image 13
Creating Image 198 as 8x12 subimage of Image 13
Creating Image 199 as 8x12 subimage of Image 13
Creating Image 200 as 8x12 subimage of Image 13
Creating Image 201 as 8x12 subimage of Image 13
Creating Image 202 as 8x12 subimage of Image 13
Creating Image 203 as 8x12 subimage of Image 13
Creating Image 204 as 8x12 subimage of Image 13
Creating Image 205 as 8x12 subimage of Image 13
Creating Image 206 as 8x12 subimage of Image 13
Creating Image 207 as 8x12 subimage of Image 13
Creating Image 208 as 8x12 subimage of Image 13
Creating Image 209 as 8x12 subimage of Image 13
Creating Image 210 as 8x12 subimage of Image 13
Creating Image 211 as 8x12 subimage of Image 13
Creating Image 212 as 8x12 subimage of Image 13
Creating Image 213 as 8x12 subimage of Image 13
Creating Image 214 as 8x12 subimage of Image 13
Creating Image 215 as 8x12 subimage of Image 13
Creating Image 216 as 8x12 subimage of Image 13
Creating Image 217 as 8x12 subimage of Image 13
Creating Image 218 as 8x12 subimage of Image 13
Creating Image 219 as 8x12 subimage of Image 13
Creating Image 220 as 8x12 subimage of Image 13
Creating Image 221 as 8x12 subimage of Image 13
Creating Image 222 as 8x12 subimage of Image 13
Creating Image 223 as 8x12 subimage of Image 13
Creating Image 224 as 8x12 subimage of Image 13
Creating Image 225 as 8x12 subimage of Image 13
Creating Image 226 as 8x12 subimage of Image 13
Creating Image 227 as 8x12 subimage of Image 13
Creating Image 228 as 8x12 subimage of Image 13
Creating Image 229 as 8x12 subimage of Image 13
Creating Image 230 as 8x12 subimage of Image 13
Creating Image 231 as 8x12 subimage of Image 13
Creating Image 232 as 8x12 subimage of Image 13
Creating Image 233 as 8x12 subimage of Image 13
Creating Image 234 as 8x12 subimage of Image 13
Creating Image 235 as 8x12 subimage of Image 13
Creating Image 236 as 8x12 subimage of Image 13
Creating Image 237 as 8x12 subimage of Image 13
Creating Image 238 as 8x12 subimage of Image 13
Creating Image 239 as 8x12 subimage of Image 13
Creating Image 240 as 8x12 subimage of Image 13
Creating Image 241 as 8x12 subimage of Image 13
Creating Image 242 as 8x12 subimage of Image 13
Creating Image 243 as 8x12 subimage of Image 13
Creating Image 244 as 8x12 subimage of Image 13
Creating Image 245 as 8x12 subimage of Image 13
Creating Image 246 as 8x12 subimage of Image 13
Creating Image 247 as 8x12 subimage of Image 13
Creating Image 248 as 8x12 subimage of Image 13
Creating Image 249 as 8x12 subimage of Image 13
Creating Image 250 as 8x12 subimage of Image 13
Creating Image 251 as 8x12 subimage of Image 13
Creating Image 252 as 8x12 subimage of Image 13
Creating Image 253 as 8x12 subimage of Image 13
Creating Image 254 as 8x12 subimage of Image 13
Creating Image 255 as 8x12 subimage of Image 13
Creating Image 256 as 8x12 subimage of Image 13
Creating Image 257 as 8x12 subimage of Image 13
Creating Image 258 as 8x12 subimage of Image 13
Creating Image 259 as 8x12 subimage of Image 13
Creating Image 260 as 8x12 subimage of Image 13
Creating Image 261 as 8x12 subimage of Image 13
Creating Image 262 as 8x12 subimage of Image 13
Creating Image 263 as 8x12 subimage of Image 13
Creating Image 264 as 8x12 subimage of Image 13
Creating Image 265 as 8x12 subimage of Image 13
Creating Image 266 as 8x12 subimage of Image 13
Creating Image 267 as 8x12 subimage of Image 13
Creating Image 268 as 8x12 subimage of Image 13
Creating Image 269 as 8x12 subimage of Image 13
Decrementing Image 13 refcount, new: 257
Decrementing Image 13 refcount, new: 256
Incrementing Font 0 refcount, new: 1
Calling game()

JS Error: game.js:6 - Error: Hello, mixer!
Shutting down map engine
Shutting down persons manager
Shutting down input
Shutting down Duktape
Shutting down Dyad
Shutting down spriteset manager
  Objects created: 0
  Cache hits: 0
Shutting down Audialis
Decrementing Mixer 0 refcount, new: 0
Mixer 0 no longer in use, deallocating
engine(37980,0x108c34000) malloc: *** error for object 0x7febeb812200: pointer being freed was not allocated
*** set a breakpoint in malloc_error_break to debug
Abort trap: 6
Title: Re: minisphere 1.5.3
Post by: Fat Cerberus on July 20, 2015, 05:34:10 pm
Okay, so in this case it's crashing trying to free the main mixer.  So why on earth does the stack contain no minisphere code whatsoever...?  The only thing I can think of is that some sort of exception is being raised and Allegro's shim for main() is catching it, obscuring the real source of the bug.

This is quite a stumper.
Title: Re: minisphere 1.5.3
Post by: Flying Jester on July 20, 2015, 05:41:18 pm
Or it's on a different thread.
Title: Re: minisphere 1.5.3
Post by: Fat Cerberus on July 20, 2015, 05:54:56 pm

Or it's on a different thread.


That's what I suspected at first too, but the stack says otherwise: there's a call_user_main() in there, which wouldn't make sense for a worker thread, and I also see an exit() in there, so it's running atexit stuff.  Looks like the main thread to me.

Wait a minute!  I seem to remember that Duktape calls abort() in the case of unrecoverable errors.  Allegro isn't throwing any asserts even though it's clearly a debug build (Allegro source shown in lldb), which means I may be barking up the wrong tree.  CoffeeScript triggered a recursion error during startup, while it works normally here.  So something wonky may actually be going on in Duktape...

NEO, add this define to the top of duktape.c:
Code: (c) [Select]
#define DUK_OPT_ASSERTIONS


And recompile.  See if any asserts fail.
Title: Re: minisphere 1.5.3
Post by: N E O on July 20, 2015, 06:21:26 pm
So something wonky may actually be going on in Duktape...

NEO, add this define to the top of duktape.c:
Code: (c) [Select]
#define DUK_OPT_ASSERTIONS


And recompile.  See if any asserts fail.


The console output doesn't show any failed asserts, no.
Title: Re: minisphere 1.5.3
Post by: Fat Cerberus on July 21, 2015, 12:46:07 am
Hm, well it was worth a shot at least.  Having studied your backtrace AGAIN (it was very illuminating, thanks!), I notice an exit() call in there.  So something, somewhere is calling, I assume abort() in response to the crash, and that could be why main is no longer on the stack (do exit()/abort() unwind the stack?).  That's why I suspected Duktape, because I know it will do so in case of an unrecoverable error.

I find it interesting that it still crashes in response to canceling the file dialog, but in a different spot than the crash on shutdown.  I'm curious now, will it run normally if you give it an actual game (not a Hello World stub) to run and only crash on shutdown, or would issues show up sooner...

By the way, not related to the crash, but could you add a define for DUK_OPT_DEEP_C_STACK to duktape.c and let me know if CoffeeScript starts working again (i.e. the RangeError disappears)?
Title: Re: minisphere 1.5.3
Post by: N E O on July 21, 2015, 04:32:19 pm

By the way, not related to the crash, but could you add a define for DUK_OPT_DEEP_C_STACK to duktape.c and let me know if CoffeeScript starts working again (i.e. the RangeError disappears)?

The RangeError does indeed disappear, yes.
Title: Re: minisphere 1.5.3
Post by: Fat Cerberus on July 21, 2015, 04:45:17 pm
Thanks. :D

Now if only I could figure out what's causing these crashes...  I won't rule out that the bug is on my end.  I know I've encountered similar issues in the past where things that should fail end up working on Windows preventing me from reproducing them.  For instance, a while ago DaVince discovered a crash in 1.2.2 that I couldn't reproduce, caused by a double call to fclose().  This has the same effect as a double free in Linux but in Windows it doesn't raise any red flags whatsoever--it succeeds without complaint.  I only figured it out after poring over the RawFile code for a while.
Title: Re: minisphere 1.5.3
Post by: N E O on July 21, 2015, 10:44:21 pm

Thanks. :D

Now if only I could figure out what's causing these crashes...  I won't rule out that the bug is on my end.  I know I've encountered similar issues in the past where things that should fail end up working on Windows preventing me from reproducing them.  For instance, a while ago DaVince discovered a crash in 1.2.2 that I couldn't reproduce, caused by a double call to fclose().  This has the same effect as a double free in Linux but in Windows it doesn't raise any red flags whatsoever--it succeeds without complaint.  I only figured it out after poring over the RawFile code for a while.


If I could understand Allegro better I would help poring over Audialis, since if it likely ends up being a double free to allegro_audio it's almost certainly there.

*sigh* I am so far behind in my independent programming studies and now currently lack motivation to learn anything useful :'(
Title: Re: minisphere 1.6.0
Post by: Fat Cerberus on July 27, 2015, 05:04:09 pm
minisphere 1.6.0 is released.  See changelog for details.

The debugging feature should work, as it just uses the Duktape debug API.  A client needs to be written using Duktape's remote debug protocol to take advantage of it, however:
https://github.com/svaarala/duktape/blob/master/doc/debugger.rst
Title: Re: minisphere 1.6.1
Post by: Fat Cerberus on July 31, 2015, 02:44:27 am
1.6.1 fixes a bug in the Sockets API where listening sockets wouldn't accept connections from IPv4 clients (including local clients, i.e. those connecting to 127.0.0.1).

I can also definitively say the debugging support in 1.6 works great.  I got a proof of concept plugin working in Sphere Studio that just sends a "resume" message to the engine.  The game runs upon receiving this message, so everything's in order so far!  Now to implement full debugger support in Sphere Studio... ;D
Title: Re: minisphere 1.6.1
Post by: Fat Cerberus on August 02, 2015, 03:04:24 am
@NEO, regarding Audialis:
Looks like minisphere isn't at fault here after all, it's an Allegro bug:
https://www.allegro.cc/forums/thread/615559/1015654

Everyone else: The debugger is coming along nicely. :)  Pretty soon we'll all be able to debug Sphere games like any other app!
Title: Re: minisphere 1.6.1
Post by: Fat Cerberus on August 02, 2015, 10:34:04 pm
NEO, could you try building the latest Allegro from the github mirror and let me know if the crashes are gone?
Title: Re: minisphere 1.6.1
Post by: N E O on August 04, 2015, 08:12:48 pm
(building Allegro from Git on OS X for future reference: cd allegro5; git pull; mkdir build; cd build; cmake ..; sudo make install)


NEO, could you try building the latest Allegro from the github mirror and let me know if the crashes are gone?


Rebuilt Allegro, pulled latest minisphere, removed MNG stuff as usual (I'll eventually go find libmng and build it to resolve this bit), recompiled. Ran my one-liner and exits clean! Ran and canceled choosing a game and exits clean! Ran CellarRush.spk and got an error that it failed to load sound file '../music/dungeon-v2.ogg' and exits clean!

For reference, building Allegro has consistently given me warnings about inability to build in loading of many audio formats like vorbis and speed.

Alright! Except for me advancing on the MNG front and figuring out how to fix audio building without building all the libs from scratch (it'd be nice if homebrew would update Allegro) it seems Sphere on Mac can indeed progress further! Now to resume my Pang96 conversion... ;)
Title: Re: minisphere 1.6.1
Post by: Fat Cerberus on August 04, 2015, 08:15:30 pm
Allegro's handling of audio format libs when building is dumb.  There are certain flags you have to set in CMake to get it to work right.  It's easier I've found to use cmake-gui.
Title: Re: minisphere 1.6.1
Post by: Fat Cerberus on August 04, 2015, 09:07:53 pm
Follow-up: Do you have the necessary libraries installed?  For full audio support you need:

* libvorbis
* libvorbisfile
* libogg
* dumb
* libflac
Title: Re: minisphere 1.6.1
Post by: HopeMetal on August 05, 2015, 06:59:37 am
I downloaded this a few days ago and I could run Sphere Studio just fine (even though I ran into problems trying to actually make anything), but now it doesn't even launch at all. When I try to run it just doesn't open. Help?
Title: Re: minisphere 1.6.1
Post by: Fat Cerberus on August 05, 2015, 10:04:03 am
Something probably got corrupted.  Delete MyDocuments/SphereStudio/Settings and try again.
Title: Re: minisphere 1.6.1
Post by: Fat Cerberus on August 05, 2015, 03:14:41 pm
Everyone: Next release will change the debugger semantics (allowing a debugger to be attached at any time) and include a debug-enabled build of Sphere Studio. :)

Edit: Also in the pipeline: ArrayBuffer support.  I bit the bullet and procured the latest build of Duktape to enable this.  From what I've seen, the Duktape master is extremely well-tested, so I'm not too concerned about catastrophic bugs there. :)  There are also supposed to be some performance improvements in the latest builds!
Title: Re: minisphere 1.6.1
Post by: HopeMetal on August 06, 2015, 04:42:53 am

Something probably got corrupted.  Delete MyDocuments/SphereStudio/Settings and try again.

Deleting the whole folder did the trick.

EDIT: But now when I try to add files to my project, it gives me unhandled exception errors about "illegal characters in path" and "Object reference not set to an instance of an object". When I try adding a spriteset it says "Can't load spriteset: ?.rss".
Title: Re: minisphere 1.6.1
Post by: Fat Cerberus on August 06, 2015, 10:04:11 am
The spriteset creation is probably a bug caused by my recent changes (it's going through a bit of upheaval at the moment), I'll look into it.  Not sure about the illegal filename characters though...  That seems weird.
Title: Re: minisphere 1.6.1
Post by: HopeMetal on August 06, 2015, 02:38:51 pm

The spriteset creation is probably a bug caused by my recent changes (it's going through a bit of upheaval at the moment), I'll look into it.  Not sure about the illegal filename characters though...  That seems weird.

I guess I'll use the editor provided with Sphere 1.5 for now. I just need to figure out how to get it to work with Minisphere.
Title: Re: minisphere 1.6.1
Post by: Fat Cerberus on August 06, 2015, 02:49:39 pm
You MIGHT be able to use a stock installation of Sphere 1.5 and copy minisphere's engine.exe over the 1.5 one.  That should work.
Title: Re: minisphere 1.6.1
Post by: Fat Cerberus on August 07, 2015, 01:27:39 am
Hey guys, see the screenshot below for what you have to look forward to! ;D

Having a Sphere debugger is so awesome.
Title: Re: minisphere 1.6.1
Post by: Radnen on August 07, 2015, 10:39:15 pm
Literally the very best thing from a new Sphere engine and editor combination. This is just really awesome! :)
Title: Re: minisphere 1.6.1
Post by: Fat Cerberus on August 08, 2015, 12:40:19 pm
The new Duktape build also seems to be a lot faster--game startup is now on par with Sphere 1.5 now, even for large games like Aquatis which used to take up to 5 seconds to compile all the scripts.
Title: Re: minisphere 1.6.1
Post by: DaVince on August 08, 2015, 08:13:57 pm
Amazing. That looks super useful.
Title: Re: minisphere 1.6.1
Post by: Fat Cerberus on August 11, 2015, 02:10:02 pm
minisphere 1.7 is out!

So... anyone care to test out the new debugger? ;)
Title: Re: minisphere 1.7.1
Post by: Fat Cerberus on August 12, 2015, 11:47:37 pm
Well, that didn't take long... I had to bump to 1.7.1 because I forgot to include a Duktape fix for improper breakpoint triggering in 1.7.0 (breakpoints inside if statements would trigger even if the condition itself wasn't hit).
Title: Re: minisphere 1.7.1
Post by: Radnen on August 13, 2015, 01:55:28 am
I tried out the debugger and I must say it's rather well done! I'm impressed. :) It doesn't handle infinite loops though, like a main game loop. But for certain function calls it's good to step through them and see what they do.

The packager is neat too. This is feeling like a real editor now. This editor with minisphere + debug is far more robust of a tool than ever before.
Title: Re: minisphere 1.7.1
Post by: Fat Cerberus on August 13, 2015, 01:58:29 am
If you step over a MapEngine call, it won't return to the debugger because MapEngine() runs its own loop.  If you use the analogue.js included with 1.7 you can set breakpoints inside map scripts though. :)

I don't know if you noticed, but in the variable viewer there's also an "eval" textbox... you can enter code there while execution is paused to inject stuff.  Whatever you enter will be evaluated within the current scope so that you can modify local variables, etc.
Title: Re: minisphere 1.7.1
Post by: Radnen on August 13, 2015, 03:10:11 am

If you step over a MapEngine call, it won't return to the debugger because MapEngine() runs its own loop.  If you use the analogue.js included with 1.7 you can set breakpoints inside map scripts though. :)


Yeah, that makes sense. I'll definitely try out the eval, it was very helpful to have that in the .NET debugger and so to do that in minisphere - that's just wow, that's amazing!

BTW +1 for analogue shoutout, this is not something you could have done easily in the map editor.
Title: Re: minisphere 1.7.1
Post by: Fat Cerberus on August 13, 2015, 12:48:05 pm
Note that the analogue debugging is only possible because I modified analogue.js to use EvaluateScript() to pull in the mapscripts.  This is incompatible with Sphere 1.x because EvaluateScript() doesn't return anything in that engine for some reason.

Unfortunately this change was necessary because otherwise Duktape doesn't assign a filename to the script and therefore breakpoints won't work.  Overall I don't think it's a big deal though--I would say the vanilla engine is well and truly obsolete at this point. ;)
Title: Re: minisphere 1.7.1
Post by: Fat Cerberus on August 14, 2015, 01:50:39 am
Here's a screenshot for another neat debug feature I added at the last minute: With the debugger attached, Print() calls are intercepted and sent to the debugger.
Title: Re: minisphere 1.7.1
Post by: Eggbertx on August 14, 2015, 03:27:40 pm
This may have been covered already, but there are 32 pages. minisphere doesn't accept constant (const) variables?
Also, it doesn't really look like the engine doesn't really say much about what went wrong. It just says "Parse error"
Title: Re: minisphere 1.7.1
Post by: Fat Cerberus on August 14, 2015, 03:29:40 pm
Yeah, "parse error" is all Duktape will give.  Not much I can do about that.  And yes, const isn't ES5 compliant so it's not supported.  The Duktape developer is working on adding ES6 features though, so hopefully it'll make it in soon.  In any case, it's mentioned in the readme.
Title: Re: minisphere 1.7.2
Post by: Fat Cerberus on August 15, 2015, 02:19:54 am
1.7.2 is posted, fixing a few bugs in Sphere Studio and making debugging local-only, avoiding the Windows Firewall prompt.  Oh, and there's a new stack viewer too. :D
Title: Re: minisphere 1.7.2
Post by: Fat Cerberus on August 16, 2015, 05:23:41 pm
So who thought minisphere would ever be this successful?  Least of all me.  In less than a year we now have a 99% compatible replacement for the vanilla engine, a bunch of new Sphere 2.0 stuff and even a stepping debugger, something I think Sphere has needed for a long time.  And all this from an experiment that I thought would never get off the ground...

At this point, at v1.7 I feel I can call minisphere fully mature.  Which means, I think it's time to get started on that JS map engine. :). Now more than ever, with a full debugger, being able to step into map engine code will be very useful, I think.
Title: Re: minisphere 1.7.4
Post by: Fat Cerberus on August 18, 2015, 02:56:53 am
Well, it turns out right-click New was completely broken in recent builds of Sphere Studio (those invalid filename errors someone mentioned recently)... I fixed it.
Title: Re: minisphere 1.7.4
Post by: N E O on August 18, 2015, 01:00:38 pm
we now have a 99% compatible replacement for the vanilla engine


What's the missing 1%?
Title: Re: minisphere 1.7.4
Post by: Fat Cerberus on August 18, 2015, 01:03:09 pm
Two things mainly: stricter argument type checking (this is an issue in rare cases), and a few lesser-used APIs, such as Complex().
Title: Re: minisphere 1.7.4
Post by: N E O on August 18, 2015, 01:37:28 pm
I'm tempted to ask you to fork minisphere once that's done so that one version has only the vanilla stuff for 1.5/1.6 (i.e. disable all the experimental/minisphere-specific API), we'd commit that to vanilla Sphere, then officially recognize minisphere as the vanilla successor and all the experimental/ms-specific stuff would be considered "proposed additions to Sphere 2.0" or "draft status" (using W3C terminology), then it's an arms race between the big three Sphere engines.

In this scenario, for lack of a better analogy minisphere becomes WebKit, TurboSphere becomes Gecko, SSFML becomes IE/Edge, YASE/sphereclone (http://forums.spheredev.org/index.php/topic,72.0.html) might be Opera/Presto unless casiotone decides to resume work on it or it gets folded into another engine project, the various web-sphere implementations are still web-based, and all other attempts at Sphere-compatible engines still exist but we haven't heard from them in a while.
Title: Re: minisphere 1.7.4
Post by: Fat Cerberus on August 18, 2015, 01:51:25 pm
I don't know, at this point I feel like that would just be needlessly extending the vanilla engine's life for no real gain.  If I were to add the missing primitives the engine would be 100% backward compatible as-is (save some audio formats), no need to make a build with removed functionality.  We WANT to get new users to use the new stuff, and having separate stable and unstable/experimental versions might scare them off of doing so.  In fact the additional bells and whistles are the biggest reason I recommend minisphere over vanilla--a build limited to just the 1.x APIs would Have no real advantage over the original engine, IMO.  Duktape is faster, yes, but not enough to make-or-break.

It's tempting to want to wait until Sphere 2.0 has stabilized to commit to anything, but part of the reason I took the initiative with minisphere in the first place was because that process was going nowhere--note the sorry state Pegasus ended up in.

I think our best bet is to embrace the extension system I already have implemented and, instead of standardizing "Sphere 2.0" wholesale, we work on standardizing the individual components.  So you would have a standard for Audialis, one for Galileo, etc., each with a unique extension string exposed through GetExtensions() and they would all be Sphere 2.0 components.  This is how the Ecmascript specs are built, and the process seems to work very well.
Title: Re: minisphere 1.7.4
Post by: Radnen on August 18, 2015, 07:48:09 pm

It's tempting to want to wait until Sphere 2.0 has stabilized to commit to anything, but part of the reason I took the initiative with minisphere in the first place was because that process was going nowhere--note the sorry state Pegasus ended up in.


Yeah, that's why I made Sphere Studio. There was a time when Vexo (man, what happened to these people :() said every year someone comes up with the idea of a new IDE and it always fails. I was thinking if I used a proper language and environment, say C# and WinForms I can make an IDE three times faster than anyone else (imagine doing it - alone - in C or C++!) and I'd have managed, clean code to work off of. .NET was built around the concept of being a desktop application developers language, or rather for Software Engineering projects in general.
Title: Re: minisphere 1.7.4
Post by: N E O on August 18, 2015, 09:50:05 pm

I don't know, at this point I feel like that would just be needlessly extending the vanilla engine's life for no real gain.  If I were to add the missing primitives the engine would be 100% backward compatible as-is (save some audio formats), no need to make a build with removed functionality.  We WANT to get new users to use the new stuff, and having separate stable and unstable/experimental versions might scare them off of doing so.  In fact the additional bells and whistles are the biggest reason I recommend minisphere over vanilla--a build limited to just the 1.x APIs would Have no real advantage over the original engine, IMO.  Duktape is faster, yes, but not enough to make-or-break.

It's tempting to want to wait until Sphere 2.0 has stabilized to commit to anything, but part of the reason I took the initiative with minisphere in the first place was because that process was going nowhere--note the sorry state Pegasus ended up in.

For all intents and purposes, the primary advantage a vanilla API minisphere would have over classic Sphere is the existence of reliable modern dependencies and cross-platform consistencies across modern systems. minisphere comes with Duktape bundled and only requires the easy-to-package Allegro, and is guaranteed to work on all three common modern systems (and probably will work on Windows < 7, OS X < 10.9, various older Linuxes, maybe the BSDs); Classic requires a slew of first-party dependencies that may or may not bundle/compile/package/etc cleanly and easily on modern systems (I'm sure y'all remember the PITA modernizing audiere was), and is only guaranteed to work on older Windows (I don't remember if 1.5 worked easily on >=Vista or not), OS X < 10.7 with the separate dependencies bundle, and maybe some Linuxes.

But yes, I understand the desire to "get new users to use the new stuff;" remember I was a web developer trying to convince clients to modernize sites to support Firefox & Chrome back when IE6 was still a thing ;)


I think our best bet is to embrace the extension system I already have implemented and, instead of standardizing "Sphere 2.0" wholesale, we work on standardizing the individual components.  So you would have a standard for Audialis, one for Galileo, etc., each with a unique extension string exposed through GetExtensions() and they would all be Sphere 2.0 components.  This is how the Ecmascript specs are built, and the process seems to work very well.

Ecmascript, CSS3, and HTML5 use this modular draft system, and I agree with its methodology. The doc for GetExtensions says the strings returned are mostly self-explanatory, but honestly I feel the new ones (basically any string that's not "sphere-legacy-api") need some sort of explanation, and I'd probably like to see some sectioned, e.g. sphere-audialis sectioned into sphere-audialis, sphere-audialis-wav, sphere-audialis-mp3, sphere-audialis-midi, sphere-audialis-ogg, ..., as R/W capabilities warrant. I assume the preferred way to check for certain functionality is (GetExtensions().indexOf(theCapability)!==-1)  ?

The doc for GetVersion says minisphere 1.4 and needs the version number updated, FYI.
Title: Re: minisphere 1.7.4
Post by: Fat Cerberus on August 18, 2015, 10:47:11 pm
Does it?  ill have to fix it.  I make a point to update the GetVersionString() return value every major release, but may have missed an annotation elsewhere.  The actual function should return the correct API version (2.0) in any case.

indexOf is the best way to check for extensions, yes.

But anyway, I don't think actively stripping out functionality helps anyone when the engine is already compatible with vanilla.  Vanilla compatibility is neatly encapsulated by the sphere-legacy-api extension, if you need to code against the new stuff you check for the appropriate extension and either degrade gracefully or fail outright.  That's why the system exists.

Speaking of which, I don't know that it's a good idea to overly compartmentalize the extension system as you're suggesting.  Extensions are shorthand for a set of APIs, supported formats don't really need dedicated extensions as more formats might be supported via plugins or whatnot, a la TurboSphere--and depending on the backend used, may even change between platforms...

As for documentation, again, each extension string represents an API.  So to find out what each represents if it's unclear, you have to read the API reference. ;)  Both Galileo and Audialis are covered via short descriptions and function documentation.  Even SphereFS is explained in that document.

Overall I just think that releasing a stripped-down engine is akin to saying "here's a modernized IE6, we know you're not ready for all the new technologies yet but at least it's new, right?"  Maybe that's just me though.
Title: Re: minisphere 1.7.4
Post by: N E O on August 19, 2015, 01:19:55 am

But anyway, I don't think actively stripping out functionality helps anyone when the engine is already compatible with vanilla.  Vanilla compatibility is neatly encapsulated by the sphere-legacy-api extension, if you need to code against the new stuff you check for the appropriate extension and either degrade gracefully or fail outright.  That's why the system exists.

[ ... ]

Overall I just think that releasing a stripped-down engine is akin to saying "here's a modernized IE6, we know you're not ready for all the new technologies yet but at least it's new, right?"  Maybe that's just me though.


I understand your approach better now. I was of an old mindset of "test if GetVersion() > X for Y functionality" to do stuff and forgot for a while that a modular extensions system like yours and testing for extension existence is indeed almost always the better option; it's the core tenet of Modernizr, among other libraries, and Crockford preaches about the methodology so hard in JS: The Good Parts that I really should have known better.

...crap. That it took me this long to put 2 & 2 together about this means the depression I seem to have is getting worse; the fact that I'm apparently losing focus and acuity in THIS aspect of my life is a pretty bad sign for me.
Title: Re: minisphere 1.7.4
Post by: Fat Cerberus on August 19, 2015, 02:17:14 am
Yeah, that's pretty much it.  Technically GetVersion() never has to return anything > 2.0 now since 2.0 implies the existence of GetExtensions()--and if THAT exists, you should be using that to test for feature support beyond the vanilla subset.
Title: Re: minisphere 1.7.4
Post by: Fat Cerberus on August 19, 2015, 01:53:29 pm
So I'm working on a modification to Duktape that will allow minisphere to break into the debugger if an error is thrown.  Attached is a screenshot of the dialog which would be displayed by Sphere Studio if this is successfully implemented:
Title: Re: minisphere 1.7.4
Post by: Fat Cerberus on August 20, 2015, 09:53:55 am
I think I want to make a command line debugger for minisphere.  Being able to do it in Sphere Studio is great (I always prefer an IDE for debugging), but this would allow debugging on any platform minisphere runs on, not just windows.  Especially if I coded it in C.
Title: Re: minisphere 1.7.4
Post by: Eggbertx on August 20, 2015, 02:57:49 pm
How do I send text to the console?
Title: Re: minisphere 1.7.4
Post by: Fat Cerberus on August 20, 2015, 03:39:19 pm
Code: (javascript) [Select]
Print("A pig ate everything at 8:12");


While debugging, this will also send it to the Sphere Studio console.
Title: Re: minisphere 1.7.5
Post by: Fat Cerberus on August 22, 2015, 01:24:52 am
1.7.5 is now available for download with the previously mentioned debugger enhancement allowing the debugger to pause execution when an error is thrown.  Very useful!
Title: Re: minisphere 1.7.6
Post by: Fat Cerberus on August 29, 2015, 12:49:15 am
minisphere 1.7.6 is up.  The redist engine now has JS debugging disabled as having it enabled hurt performance even when not actively debugging.  Debugging now requires minisphere Console to be used.

Sphere Studio has also been updated with the latest bug fixes and features, notably the new entity view implemented in the map editor by Radnen.
Title: Re: minisphere 1.7.7
Post by: Fat Cerberus on September 01, 2015, 12:03:30 pm
Okay, 1.7.7 is up, fixing misbehavior when infinite recursion is encountered--"double error" for a normal run, or an outright crash with the debugger attached.  Now it properly throws a RangeError in this case, which will be caught by the debugger.
Title: Re: minisphere 1.7.11
Post by: Fat Cerberus on September 10, 2015, 09:12:00 pm
I decided for minisphere 2.0, I'm going to deprecate ByteArray.  It will still be retained in the API for backwards compatibility, but all the new APIs (such as the new Sockets API) are being updated to use ArrayBuffers and TypedArrays instead.

I'm also going to try my hand at writing a new map engine in JavaScript.  Despite my misgivings in the past, I think this will ultimately be a good thing to have, if only because you'll be able to step into map engine code while debugging.  As with ByteArrays above, the classic map engine will be retained with its API fully intact.
Title: Re: minisphere 1.7.11
Post by: Fat Cerberus on September 19, 2015, 12:08:19 am
Good news: minisphere 2.0 will support const.  Well, in the same capacity as Sphere 1.5 does anyway (i.e. it's a synonym for var).  This improves legacy compatibility even further. :)
Title: Re: minisphere 1.7.11
Post by: Fat Cerberus on September 19, 2015, 01:47:06 am
Let me just say for the record here that SCons is a hellish nightmare to work with.  Things that should be simple like pulling in sources from minisphere to use in Cell cause no end of grief under SCons because it doesn't make a clear distinction between source and object files and treats one as a stand-in for the other.  That is, if you use the same source file in two programs, it compiles it once and tries to link the same object file into both programs.  And then if you try to use different compiler options between the two, SCons throws up its hands and gives up ("Two environments with different options for same target").

It doesn't help either that the SCons documentation is absolutely atrocious.  It took me forever just to find (on an obscure Stack Overflow question) that the solution was not to pass the .c file to Program() directly, but use the Object() method like so:

Code: (python) [Select]
cell_sources = [
Object("duktape", "../msphere/duktape.c"),
Object("vector", "../msphere/vector.c"),
"engine.c",
"main.c"  # <-- what this really means, to SCons, is "the object file created from main.c"
]


Ugh.

edit: GAAAAAAAAAAAAAAAHHHHHHHHHHHH *DBZ powerup* *goes Super Saiyan*
Code: [Select]
$ scons cell
scons: Reading SConscript files ...
scons: done reading SConscript files.
scons: Building targets ...
scons: building associated VariantDir targets: obj/cell
scons: *** [obj/cell/engine.o] Implicit dependency `obj/cell/duktape.h' not found, needed by target `obj/cell/engine.o'.
scons: building terminated because of errors.


That didn't work either.  At all.  Seriously, whoever wrote this piece of software should be drug out into the street and shot.

edit 2: So it started building properly again for no apparent reason.  Well, I'm not going to complain, but it doesn't do much for my faith in this build system.  I wonder if CMake is any better.  In any case, SCons provides a good baseline of what not to do in Cell. :P
Title: Re: minisphere 1.7.11
Post by: Flying Jester on September 19, 2015, 06:37:55 pm
Chad Austin is in the credits for SCons. That's how I originally found it.

The docs are bad, mostly because SCons wiki was destroyed by spambots a few years ago. I usually use the archive.org'ed version of it, since it was never properly rebuilt.

SCons doesn't make any distinctions about what anything is. Everything has a way of being created, and may be needed for something else. If you only care about compiling that C file, then you pass in that C file. Since you care about more than that (compiler options), what you care about is the resulting object files. This is how any dependency driven build system works.

An alternative (I suspect the intended way) is to use static libraries, since using the static library builder lets you do what you want--different options on the same source to create different object files. In general, static libraries stand in for individual object files in SCons when you don't want to build something in just one step.

When you hit implicit dependency issues, try nuking the .sconsign and scons DB files.
Title: Re: minisphere 1.7.11
Post by: Fat Cerberus on September 19, 2015, 06:47:24 pm
Okay, but if I use the same .c file in two programs, why does SCons think its a reasonable default to only generate a single object file for that source, even though the two programs may (and probably do) need to be compiled with different options?  That's what I meant by using sources as a stand-in for objects--if I tell it to build a program from main.c, SCons interprets that as "build a program from the object file created from main.c"--and any reference anywhere (even in different environments!) to main.c refers back to that exact same object file.  It makes no sense and I feel like I have to jump through hoops (using Object builders, etc.) just to get it to do something sane.
Title: Re: minisphere 1.7.11
Post by: Flying Jester on September 19, 2015, 06:58:22 pm
Referring directly to main.c when using either the shared library or program builders basically means you want any object file of that source. When you mean a specific object file, then you must specify that specific object file.

Basically, it's just a shortcut that simplifies things in 99% of the uses. This way you don't need to specify the object builder for every single source file, and managed each object file through to the library or program builder. In this case, you don't want to use that shortcut, and unfortunately the long-hand way is kind of obscure.

The other answer is that it's kind of bad practice to compile the exact same source with functionally different compiler options. But that's kind of a bad answer, since I know a lot of projects (both C and C++) do it.

I usually do this sort of thing (mostly when I want different preprocessor directives) by creating a C or C++ file for each use of the single source file, and including it from those. I never compile the main source directly, but instead the files that included it. That also keeps as much of the preprocessr usage out of the build system as possible, which I consider a good thing. The down side is that I'm including actual source files in other source files, which is a little unusual. But the up side is that it is clear what options are being specified from the source itself, and it also makes supporting multiple build systems easier.
Title: Re: minisphere 1.7.11
Post by: Fat Cerberus on September 22, 2015, 03:11:46 am
So I may have gotten spoiled by Allegro.  I had to pretty much reimplement its entire path builder API from scratch for Cell because it was very powerful but there was no way I was depending on Allegro for a command-line tool.  On the upside, my custom path routines now have better canonization than Allegro, I can have it collapse upstream (../) hops which Allegro's canonizer won't do (it's not symlink-friendly) but is needed when working with, e.g. SPKs.

Paths are represented internally as an array of hops, which makes the whole thing platform-agnostic: Only the pathname string creation needs to be changed to use a different seperator (but almost all platforms accept / so this isn't even necessary in practice).  It also makes working with paths safer since there is no risk of a buffer overflow--each hop is allocated independently using strdup().

Path building functions are one thing I always tend to miss when I'm not using .NET, so having them in C now is awesome.
Title: Re: minisphere 1.7.11
Post by: Fat Cerberus on September 27, 2015, 12:53:16 am
Cell is coming along nicely, and should be a nice addition to minisphere 2.0.  game.sgm is no longer a requirement for a Sphere project, instead you have a cell.js build script in the source directory, like this:

Code: (javascript) [Select]
function $sphere()
{
install(files("scripts/*.js", true), "scripts");
install(files("images/*.png", true), "images");
install(files("maps/*.rmp", true), "maps");
install(files("maps/*.rts", true), "maps");
install(files("spritesets/*.rss", true), "spritesets");
install(files("sounds/*.wav", true), "sounds");
install(files("sounds/*.ogg", true), "sounds");

install(sgm({
name: "Spectacles: Bruce's Story",
author: "Fat Cerberus",
description: "Follow Scott Starcross in his quest to stop the Primus.",
resolution: "320x240",
script: "main.js"
}));
}


And then, at the command line (be sure to scroll down to the bottom for something amusing ;)):

Code: [Select]

fatce@DESKTOP-AR6TT0B /d/src/spectacles-i
$ cell -p Spectacles-BS.spk
Cell v2.0-WIP Sphere Game Compiler x86
A scriptable build engine for Sphere games.
(c) 2015 Fat Cerberus

    Building 'D:/src/spectacles-i/Spectacles-BS.spk' (SPK)
    Source: 'D:/src/spectacles-i/'

Processing cell.js rule 'sphere'
Building assets...
  D:/src/spectacles-i/.cell/game.sgm
  1 asset(s) built
Installing assets...
  scripts/AIContext.js
  scripts/Battle.js
  scripts/BattleActor.js
  scripts/battleAI/HeadlessHorseAI.js
  scripts/battleAI/LumisquirrelAI.js
  scripts/battleAI/Robert1AI.js
  scripts/battleAI/Robert2AI.js
  scripts/battleAI/ScottStarcrossAI.js
  scripts/battleAI/ScottTempleAI.js
  scripts/battleAI/VictorAI.js
  scripts/BattleHUD.js
  scripts/BattleScreen.js
  scripts/BattleUnit.js
  scripts/ConditionContext.js
  scripts/Cutscenes.js
  scripts/FieldMenu.js
  scripts/gamedef/animations.js
  scripts/gamedef/battles.js
  scripts/gamedef/characters.js
  scripts/gamedef/conditions.js
  scripts/gamedef/game.js
  scripts/gamedef/items.js
  scripts/gamedef/maps.js
  scripts/gamedef/math.js
  scripts/gamedef/moveEffects.js
  scripts/gamedef/scenes.js
  scripts/gamedef/skills.js
  scripts/gamedef/stats.js
  scripts/gamedef/statuses.js
  scripts/gamedef/text.js
  scripts/gamedef/weapons.js
  scripts/GameOverScreen.js
  scripts/GrowthPopup.js
  scripts/ItemUsable.js
  scripts/lib/kh2Bar.js
  scripts/lib/SpriteImage.js
  scripts/LucidaClock.js
  scripts/main.js
  scripts/MapContext.js
  scripts/maps/main.js
  scripts/maps/Portentia.js
  scripts/maps/Testville.js
  scripts/MenuStrip.js
  scripts/MoveMenu.js
  scripts/MPGauge.js
  scripts/MPPool.js
  scripts/Party.js
  scripts/PartyMember.js
  scripts/Scrambler.js
  scripts/Session.js
  scripts/SkillUsable.js
  scripts/SpecsClient.js
  scripts/SpecsServer.js
  scripts/Stat.js
  scripts/StatusContext.js
  scripts/TargetMenu.js
  scripts/testcases/brucesStory.js
  scripts/testcases/lastLucidan.js
  scripts/TestHarness.js
  scripts/TitleScreen.js
  scripts/TurnPreview.js
  scripts/WeaponUsable.js
  images/GameOverScreen.png
  images/Logos/GameLogo.png
  images/Logos/TitleCard.png
  images/ScottSucks.png
  images/TestBattle.png
  images/TitleScreen.png
  maps/main.rmp
  maps/Portentia.rmp
  maps/Testville.rmp
  maps/Portentia.rts
  maps/TestvilleTiles.rts
  spritesets/battlers/Amanda.rss
  spritesets/battlers/Bruce.rss
  spritesets/battlers/Elysia.rss
  spritesets/battlers/H. Horse.rss
  spritesets/battlers/Justin.rss
  spritesets/battlers/Katelyn.rss
  spritesets/battlers/Lauren.rss
  spritesets/battlers/Lumisquirrel.rss
  spritesets/battlers/maggie.rss
  spritesets/battlers/maggie_hippo.rss
  spritesets/battlers/Robert.rss
  spritesets/battlers/Scott T.rss
  spritesets/battlers/Scott.rss
  spritesets/battlers/Victor.rss
  spritesets/battlers/Xemnas.rss
  spritesets/invisible.rss
  sounds/Munch.wav
  sounds/BGM/BasicInstinct.ogg
  sounds/BGM/BattleForLucida.ogg
  sounds/BGM/BruceTellsHisStory.ogg
  sounds/BGM/CreepFight.ogg
  sounds/BGM/DeathComeNearMe.ogg
  sounds/BGM/EndOfInnocence.ogg
  sounds/BGM/GameOver.ogg
  sounds/BGM/HymnOfLiberty.ogg
  sounds/BGM/LamentForBruce.ogg
  sounds/BGM/LucidaByNight.ogg
  sounds/BGM/MalmagmaManor.ogg
  sounds/BGM/ManorBoss.ogg
  sounds/BGM/MyDreamsButADropOfFuel.ogg
  sounds/BGM/MyDreamsButADropOfFuel2.ogg
  sounds/BGM/OneWorldIsNotEnough.ogg
  sounds/BGM/OurWorldCanStillBeSaved.ogg
  sounds/BGM/Portentia.ogg
  sounds/BGM/RiseOfThePrimus.ogg
  sounds/BGM/RunawayTrain.ogg
  sounds/BGM/ScottsHomecoming.ogg
  sounds/BGM/SeasonsEnd.ogg
  sounds/BGM/SpectaclesTheme.ogg
  sounds/BGM/SubfloorManor.ogg
  sounds/BGM/ThePromise.ogg
  sounds/BGM/TheQuantumEnigma.ogg
  sounds/BGM/TimeToLetGo.ogg
  game.sgm
  117 asset(s) installed
Success!

fatce@DESKTOP-AR6TT0B /d/src/spectacles-i
$ cell --explode
Cell seems to be going through some sort of transformation...
He's pumping himself up like a balloon!

    Cell says:
    "I'm quite volatile, and the slightest jolt could set me off!"

fatce@DESKTOP-AR6TT0B /d/src/spectacles-i
$ msphere Spectacles-BS.spk
Title: Re: minisphere 1.7.11
Post by: Radnen on September 27, 2015, 04:04:01 am
That's looking pretty right darn proper. :)
Title: Re: minisphere 1.7.11
Post by: Fat Cerberus on September 27, 2015, 11:43:55 am
Minor sidenote: I had to rename cell.js to Cellscript.js because otherwise cmd.exe uselessly tries to run the JS script directly when you type "cell". :-\
Title: Re: minisphere 1.7.11
Post by: Fat Cerberus on September 27, 2015, 12:40:02 pm
Now to really bring out the potential of this tool, it will need some image processing functions.  Is Corona still active or is that a dead project at this point?  I think I remember someone saying it was revived, but... If not, I'll probably just integrate libpng for image processing.

If anyone's wondering where I'm going with this, it's to support compiling spritesets and tilesets on-the-fly during the build process.  Sphere formats are a pain to edit, even with a purpose-built GUI, and would be much easier to manage I think if you could create them from bare images.  How this would work is you declare an asset in the Cellscript, like this:

Code: (javascript) [Select]
// 32x32, 8 directions, 5 frames per
var maggieSS = spriteset("src/sprites/maggie.png", 32,32, 8, 5);
install(maggieSS, "spritesets");


And then Cell would build the .rss file at compile time from the input image.  This would allow you to edit your sprites in your favorite image editor and let Cell deal with getting them into the proper format for the engine without a bunch of tedious copying and pasting into the RSS editor. ;D
Title: Re: minisphere 1.7.11
Post by: Flying Jester on September 27, 2015, 04:59:36 pm
Corona is still active. Both Chad Austin and I have branches (which are not exactly the same, although they have the same features and are binary compatible).

Most recently it's been updated to work with new TurboJpeg and MozJpeg.
Title: Re: minisphere 1.7.11
Post by: Fat Cerberus on September 28, 2015, 03:47:20 pm
Hm, I had forgotten Corona was C++.  Like minisphere, Cell is written in C so I would need to write a wrapper to be able to use it.  For now bare libpng will suffice.

Anyway, apparently the RTS tileset format doesn't support compression?  There's a "compression" flag in the header, but the RTS doc says it's unsupported.  As a result a tileset with just 256 32x32 tiles is already 1MB.

Since the flag is already there, I could add compression (either PNG or raw zlib) support easily enough (breaks backwards compat) but I'm not sure yet if it's worth the trouble--you get the zlib compression for free anyway via SPK packaging, which with Cell is ridiculously easy--just run cell -p mygame.spk.
Title: Re: minisphere 1.7.11
Post by: Flying Jester on September 28, 2015, 04:48:51 pm
Corona has a C interface, although it's 'hidden'. Corona's ABI is totally compiler agnostic--you could compile it with MingW and call it from MSVC, compile it with Solaris Studio and call it from tcc, etc. As a part of this, all functions have static versions, and all outward facing functions have no name C++ mangling.

http://chadaustin.me/cppinterface.html (http://chadaustin.me/cppinterface.html)
Title: Re: minisphere 1.7.11
Post by: Radnen on September 29, 2015, 11:57:14 am
So, here is what XNA does. XNA keeps all assets uncompressed even post pipeline (even if you fed it compressed formats like .png). The reason for this is because these games can be released in an archive format sorta like zip (or for Sphere, .spk) and so get benefited from compression. Further, when they decompress, there is no need to decompress a second time (e.g. from png to bmp). That way files can exist within a virtual file system and be streamed rather quickly into the game. This is how all great game engines work from Quake to Skyrim.

Sphere can be something like that. I was on a game developers forum and they laughed at me for having talked about releasing a game with hundreds of assets in subfolders (namely, every Sphere game not in a .spk format). BTW, spk's like other game engines shouldn't need to be compressed to the storage device - the assets should go straight from spk to RAM. But I think you may have that covered already in minisphere (;)).

So whatever processing occurs it's good to keep in mind what gets compressed or not, since it's the final compression - the .spk - which matters the most.
Title: Re: minisphere 1.7.11
Post by: Fat Cerberus on September 29, 2015, 12:06:13 pm
I take it you've looked at spk.c then. :)

But yes, that's what spk_fopen() does - it decompresses the packed file into a malloc'd buffer and any subsequent file reads simply read from that buffer.  Allowing write access (for transparency) was a bit trickier: I had to check the spk_fopen() mode string and if write access was requested, unpack the file to disk first then open that.  The end result is that minisphere's SPK handling is actually more robust than 1.5's: many games that broke under the old engine when packaged work just fine in minisphere.

That makes sense though, keeping assets uncompressed on disk.  This way too, if the SPK format gets an upgrade to use, say, LZMA compression it isn't held back by already-compressed assets being stored as-is.
Title: Re: minisphere 1.7.11
Post by: Radnen on September 29, 2015, 12:39:17 pm
The only downside I've had developing XNA games is the friggin' huge filesize on my storage device. But, that's considered standard in the industry (for all major fields whether you are a recording artist in a sound studio, a director shooting a film, or an animator or game developer). Almost nobody uses compressed formats professionally.

(btw, I love your .spk implementation, reminds me of quake).
Title: Re: minisphere 1.7.11
Post by: Fat Cerberus on September 30, 2015, 10:45:05 am
So apparently libpng v1.2 and v1.6 are mutually incompatible.  That makes things difficult, as it seems most Linux distros only provide a package for libpng 1.2, meaning my Travis build is quite thoroughly broken right now.  To get around this, I attempted to compile libpng 1.2 on Windows, but it seems that release uses an older zlib which is ALSO incompatible.  Lovely.  And I guess I'd have this same issue even if I switched to Corona, since it uses libpng...
Title: Re: minisphere 1.7.11
Post by: Flying Jester on September 30, 2015, 12:42:19 pm
No. Corona works with libpng older or newer than libpng 1.2. You just need to have compiled it for whichever libpng you have on that system.

Travis uses somewhat outdated Debian. Which is about the only place that still uses such an ancient version of libpng. My main complaint about Travis is that the system it represents is very out of date. Very few Linux users will have an environment like it.
Title: Re: minisphere 1.7.11
Post by: Fat Cerberus on September 30, 2015, 12:57:16 pm
Well, that's good to know at least.  Still stupid that the PNG group thought it was a good idea to release a 1.5/1.6 and not keep it at least somewhat source-compatible with 1.2, though.  The API is completely different.  I know PNG predates semantic versioning, but even so, that's tacky.

That said, I removed the image processing for now as it was only being used to generate tilesets, but that's not useful as the resulting tileset has no obstruction data.  I haven't come up with a solution for that yet.
Title: Re: minisphere 1.7.11
Post by: Flying Jester on September 30, 2015, 01:18:23 pm
I'm pretty sure there were a large number of versions that had both APIs. I am sure Travis is just that out of date. No library has the same API forever.

The new API is much, much better, and means that there is no forseeable reason to ever break the ABI again. It also makes the library's ABI compiler agnostic (the 1.2 and older API was not).
Title: Re: minisphere 1.7.11
Post by: Fat Cerberus on September 30, 2015, 01:36:07 pm
No, I like the new API a lot, don't get me wrong on that. :)  It's just that they should have versioned it at 2.0 to make it clearer that the upgrade wouldn't be seamless.  Keeping the major version the same implies at least mostly-backward compatibility with code from the older version, which isn't the case at all.  It certainly caught me by surprise!
Title: Re: minisphere 1.7.11
Post by: Flying Jester on September 30, 2015, 02:36:13 pm
I would consider the libpng changes mostly-backwards compatible. It only accounts for a handful of lines of code difference in Corona.
Title: Re: minisphere 1.7.11
Post by: Fat Cerberus on September 30, 2015, 02:44:10 pm
Hm, I'll look into it.  However, off topic, I found this:
https://github.com/dimensio/libpng16-deb/blob/master/README.md

Ubuntu, which Travis uses, includes 1.2.  The readme also says Debian is experimenting with adding 1.6, but there are apparently compatibility issues preventing it.  No idea why but that explains why the Travis build fails at least.
Title: Re: minisphere 2.0b1 (stable 1.7.11)
Post by: Fat Cerberus on October 02, 2015, 03:03:07 am
The first pre-release of minisphere 2.0 is now available, minisphere 2.0b1!  This includes a preview of the new Cell compiler.  To get started with Cell, place this example script in any Sphere game directory as Cellscript.js:

Code: (javascript) [Select]

function $sphere()
{
install(files("animations/*", true), "animations");
install(files("maps/*", true), "maps");
install(files("fonts/*", true), "fonts");
install(files("images/*", true), "images");
install(files("scripts/*", true), "scripts");
install(files("sounds/*", true), "sounds");
install(files("spritesets/*", true), "spritesets");
install(files("windowstyles/*", true), "windowstyles");

install(sgm({
name: "Awesome Sphere Game",
author: "Some Weird Guy",
description: "Follow the hero as he tries to get devoured by the nearest pig.",
resolution: "320x240",
script: "main.js"
}));
}


Then from the command line, switch to that game's directory and (assuming you added minisphere to the PATH during setup):
Code: [Select]

cell -p game.spk
msphere game.spk
Title: Re: minisphere 2.0b1 (stable 1.7.11)
Post by: Fat Cerberus on October 05, 2015, 02:28:47 pm
It's a good thing I really enjoy doing this or the overall deadness of the Sphere forums would be incredibly depressing after all the work I did on minisphere... :P
Title: Re: minisphere 2.0b1 (stable 1.7.11)
Post by: Radnen on October 05, 2015, 10:54:10 pm
Hopefully someday we can start marketing a nice solution to game hobbyists. I think Sphere has fallen in visibility lately and we can try to re-ignite what was there under Minisphere and my editor. Perhaps get a more advanced crowd to who'll appreciate debug tools and command line build scripts. ;)
Title: Re: minisphere 2.0b1 (stable 1.7.11)
Post by: Fat Cerberus on October 05, 2015, 11:01:05 pm
That's actually my overarching goal with the recent additions (from around 1.5 on)--to make Sphere look like a serious tool for game development, beyond the "toy" that is Sphere 1.5.  To accomplish that, I realized it needs to be more than just the engine; it doesn't matter how many awesome features I add to it, there needs to be some sort of toolchain to go with it.

If you think about it, being more than just a toy is what got Sphere on the map originally--it had JavaScript, a full programming language, while everyone else was using RPG Maker 2000 with its point and click "scripting".
Title: Re: minisphere 2.0b1 (stable 1.7.11)
Post by: DaVince on October 06, 2015, 12:10:08 am

Hopefully someday

We're pretty much already there.

Also, I remember talk about an overhauled map engine at some point? Definitely check out http://www.mapeditor.org/ - it's a fantastic cross-platform map editor I've been using since very recently which is just EXCELLENT and I think integration of its map format into minisphere would be *very* beneficial for Sphere and the community to have. And re-ignite some of its flames.

Heck, look at what already supports it (http://doc.mapeditor.org/reference/support-for-tmx-maps/). Add minisphere to that list and bam, visibility goes up. :)
Title: Re: minisphere 2.0b1 (stable 1.7.11)
Post by: Fat Cerberus on October 09, 2015, 02:54:23 am
That map editor looks awesome.  I'll see about integrating it, even if I don't support it in the engine directly I could have Cell be able to compile it to an RMP if nothing else.

Speaking of which, thanks to a recent massive overhaul of Sphere Studio's plugin system, the minisphere 2.0 GDK will have much tighter integration with the editor.  So you won't have to drop to the command line to get the full power of Cell! ;D

As for the overhauled map engine, I was considering doing one for minisphere 2.0 in JS.  I keep putting it off to do other stuff though, like the Sphere Studio overhaul, because I know it will be a huge undertaking.  Implementing the classic map engine was easily the most time-consuming part of minisphere's early development, and that was working off of the original Sphere 1.x source!
Title: Re: minisphere 2.0b1 (stable 1.7.11)
Post by: N E O on October 09, 2015, 11:41:03 am

As for the overhauled map engine, I was considering doing one for minisphere 2.0 in JS.  I keep putting it off to do other stuff though, like the Sphere Studio overhaul, because I know it will be a huge undertaking.  Implementing the classic map engine was easily the most time-consuming part of minisphere's early development, and that was working off of the original Sphere 1.x source!


Always remember the most immediate advantage of making a piece of the engine in JS is how easily portable it will be to the web versions (and likely inevitable node.js version). Once a working proof of concept is made (can probably reuse casiotone's, Radnen's, and/or my existing JS animation code (http://www.spheredev.org/fmt/) for animating tiles), THEN we can worry about optimization and stuff. If you look at the source for mine you'll see I had almost finished map stuff, but I was stuck on figuring out an efficient animated rendering method and didn't go further.

Note: in my tile animation demo section the 'A' that marks animated tiles is HTML, not canvas, so don't worry about a parsing error there if you want to save the canvas to a file.
Title: Re: minisphere 2.0b1 (stable 1.7.11)
Post by: Flying Jester on October 09, 2015, 03:56:30 pm
I found it was much, much faster and easier to write a map engine clone in JS. And I also found that, just knowing what characteristics it should have and implementing those, I came very close to creating an equivalent object-oriented API without really meaning to.

That was what prompted me to try and rewrite more of the API in JS. I found it was actually pretty easy for a lot of the API, and didn't incur much of a performance hit at all. Fortunately, SpiderMonkey is able to properly manage its own memory, so I could finally tie GPU resources to JS objects.

That was also what made me think about putting almost the entire API in script. Just expose a single ArrayBuffer that is the screen, and ArrayBuffer-like object you can post audio to, something equivalent in functionality to the RawFile object for filesystem and network access, and an object that represents input. Everything else can be done in script.
Title: Re: minisphere 2.0b1 (stable 1.7.11)
Post by: Fat Cerberus on October 09, 2015, 05:20:24 pm
That ArrayBuffer-like audio object is answered by Audialis, which as of 2.0 will support 8, 16, 24, and 32 bit audio, the last one being float.  Likewise RawFile is being deprecated in favor of the new FileStream object.

24 bit audio might be a bit awkward with ArrayBuffers though as there is no Int24Array.  Thankfully there IS a Float32Array. :)
Title: Re: minisphere 2.0b1 (stable 1.7.11)
Post by: Flying Jester on October 09, 2015, 05:35:57 pm
16-bit audio is pretty standard, though. You can assume that this is supported by just about every platform. Floating-point audio is a bit more exotic.
Title: Re: minisphere 2.0b1 (stable 1.7.11)
Post by: Fat Cerberus on October 09, 2015, 05:40:43 pm
It might still be useful to _render_ in floating-point thoigh, e.g. In the case of procedural audio.  Allegro automatically mixes it down, which is nice.
Title: Re: minisphere 2.0b1 (stable 1.7.11)
Post by: Fat Cerberus on October 11, 2015, 02:45:20 am
Thinking about the JS map engine some more, I'm beginning to think it might actually be outside the scope of the minisphere project itself.  It would be a great thing to have certainly, I just think it would be healthier as a project if it was maintained independently, instead of shipping with the engine.

The classic map engine will be kept in the minisphere 2.0 core for backwards compatibility (and as a convenient starting point for newbies), but I'd say any map engine projects going forward would be better off in the long run as independent projects.  Especially with Sphere Studio's plugin system being actually usable now, map engines don't have to be shackled to the RMP format anymore and thus there's no longer any need for the "one map engine to rule them all" model used thus far.
Title: Re: minisphere 2.0b1 (stable 1.7.11)
Post by: Fat Cerberus on October 11, 2015, 09:39:20 am
@DaVince Looked at that Tiled editor, it's pretty neat, but I'm not a big fan on the XML format for maps.  It also seems to be more complex than necessary and would require changes to the engine itself to fully support (e.g. changeable render order for tiles... weird, and kind of pointless).  And are there even any XML parsers for JS?  I'm not familiar with any if there are.
Title: Re: minisphere 2.0b1 (stable 1.7.11)
Post by: Flying Jester on October 11, 2015, 04:14:14 pm
Tiled can also output JSON, and many other formats.
Title: Re: minisphere 2.0b1 (stable 1.7.11)
Post by: Fat Cerberus on October 11, 2015, 04:48:39 pm
Do you lose the full feature set if you do though?  or would it be like saving an MS Word document in RTF, it keeps most of the formatting but you lose the more intricate stuff?
Title: Re: minisphere 2.0b2 (stable 1.7.11)
Post by: Fat Cerberus on October 12, 2015, 03:12:21 am
New minisphere 2.0 beta up with a few more engine improvements and previewing the recent Sphere Studio overhaul.  minisphere is finally starting to look like a real development platform! ;D
Title: Re: minisphere 2.0b1 (stable 1.7.11)
Post by: DaVince on October 13, 2015, 11:59:03 am
Awesome work. :)


Do you lose the full feature set if you do though?  or would it be like saving an MS Word document in RTF, it keeps most of the formatting but you lose the more intricate stuff?

From what I could see, it makes a perfect copy of all the data in the map. I tried all the different layer types, tile rotation, objects, custom properties, basically all of the major features in the editor. See attached file.
Title: Re: minisphere 2.0b2 (stable 1.7.11)
Post by: Fat Cerberus on October 14, 2015, 07:44:20 pm
I'm thinking I'm going release minisphere 2.0 as-is, with only basic file copy and SPK creation for Cell.  I started out a bit more ambitious than this, I wanted it to have various builders for maps, spritesets, etc. but that will require a bit more effort than anticipated--there's no way to specify collision data with only images, for example.  So implementing that will need more thought.

The only other improvement I might make for 2.0 is a new SGM format, probably JSON-based.  The current format is starting to show its age, especially in light of the recent SphereFS enhancements and such.
Title: Re: minisphere 2.0b2 (stable 1.7.11)
Post by: Fat Cerberus on October 17, 2015, 02:01:08 am
Want to hear something awesome?  Cell is completely compatible with Sphere 1.5. :D  I added a Cellscript to "In the Steps of the Blackfoot" (if anyone remembers, that was my first minisphere compatibility target), then used Cell to make an SPK from it and Sphere 1.5 ran the SPK with no issues.

I wasn't sure if it would work because I've had issues in the past with SPK creation where minisphere will read the package just fine but Sphere 1.5 either locks up or crashes when given the same package.  So it's good to know I finally got it right.
Title: Re: minisphere 2.0b2 (stable 1.7.11)
Post by: Radnen on October 17, 2015, 04:14:15 am
That's awesome!
Title: Re: minisphere 2.0b2 (stable 1.7.11)
Post by: Fat Cerberus on October 21, 2015, 02:57:36 pm
Progress report:
2.0 release has been delayed somewhat due to unforeseen issues.  I wanted to release it last week, but decided to implement a game.sgm replacement... and that uncovered a few shortcomings in the way the engine is currently set up.

Long story short: JS context handling is broken.  minisphere uses a single Duktape context which is created at launch and whose pointer is stored in a global variable.  This is normally fine--only one game can be run at a time.  The issues show up when a startup game is used: calling GetGameList() creates new temporary sandboxes which share the global JS environment and clobber stuff.  This in turn causes weird crashes when switching games that are difficult to pin down.  Fixing this will require a bit of refactoring, not sure how much actual work is involved yet.

Cell is basically finished though, so that's good.
Title: Re: minisphere 2.0b2 (stable 1.7.11)
Post by: Flying Jester on October 21, 2015, 03:24:30 pm
This was a huge issue in V8 and SM, too.

I for one don't really think that the game loading functions are that useful, and they go pretty hard against how modern JS engines actually work.
Title: Re: minisphere 2.0b2 (stable 1.7.11)
Post by: Fat Cerberus on October 21, 2015, 04:51:28 pm
The way I implemented it in minisphere is actually pretty clean: ExecuteGame() is simply a longjmp back into main() which shuts down everything and restarts with the new game (Duktape uses longjmp itself so it tolerates such antics).  The fix for this is easy in theory: Duktape contexts are completely independent of one another, so sandboxing is trivial.  The main issue preventing a quick fix is that API functions are also registered globally.
Title: Re: minisphere 2.0b2 (stable 1.7.11)
Post by: Fat Cerberus on October 22, 2015, 02:31:14 am
Good news: I managed to fix the bug without a huge amount of refactoring.  Which is good, because the Duktape context wasn't even the half of it: There are file-scoped static variables all over the place, etc. that would need to be sandboxed as well.

The game.sgm replacement I came up with is nifty: It's JSON-based, and all properties contained in it are exposed to script by calling GetGameManifest().  This is useful for embedding game-specific data directly in the manifest, for example debugging options, and in the future it would even be possible to store the debugger map directly in the .s2gm file (Cell currently writes it to a separate file, sourcemap.json, by analogy with MSVC's .pdb files).
Title: Re: minisphere 2.0b2 (stable 1.7.11)
Post by: Fat Cerberus on October 22, 2015, 01:21:25 pm
Just fixed the bug this morning where minisphere can't load images with the wrong file extension.  The latest Allegro release (5.1.12) includes a fix for this, but building it in Windows is a pain (its CMake script can never find all the dependencies, making it work is an exercise in frustration) so I added a workaround myself.  I'm determined to get KR running out of the box, hehe. :P

I may have mentioned this above but const works now, thanks to a quick hack by the author to add it as an alias for var.

The only thing I really have left to do is give the code another quick lookover to make sure there are no inconsistencies with the new behavior and perhaps strengthen the sandbox somewhat (it currently allows relative paths to go out of the game directory when it probably shouldn't).
Title: Re: minisphere 2.0b2 (stable 1.7.11)
Post by: Radnen on October 22, 2015, 10:22:30 pm

I may have mentioned this above but const works now, thanks to a quick hack by the author to add it as an alias for var.


That is the single best thing. Man, I think that's it then, you've reimplemented Sphere (minus a few small things?)
Title: Re: minisphere 2.0b2 (stable 1.7.11)
Post by: Fat Cerberus on October 22, 2015, 11:21:40 pm
Pretty much, yeah.  There are a handful of lesser-used APIs that I never bothered to implement such as Complex(), but otherwise nearly 100% of the Sphere API is there.  I even managed to add all the new stuff for 2.0 without changing the semantics of the legacy functions.  Of course that was only possible because Sphere doesn't have constructors. :)

Oh, and there are also a few regexes that don't work under minisphere because Duktape's parser is stricter to the ES standards than most engines.  Not much I can do there.
Title: Re: minisphere 2.0b2 (stable 1.7.11)
Post by: Fat Cerberus on October 23, 2015, 03:08:14 am
Okay, I improved the path handling in the latest minisphere commit.  Fully rooted (absolute) paths are still allowed; however, a relative path like this:

Code: (javascript) [Select]

var image = new Image("../../../image.png");


will be rejected with a very clear error message ("SphereFS sandbox violation").  This is an improvement over Sphere 1.5, which I believe just says something stupidly vague like "invalid filename".
Title: Re: minisphere 2.0b2 (stable 1.7.11)
Post by: Fat Cerberus on October 23, 2015, 10:12:18 am
Regarding compatibility, I discovered that this line in Blockman doesn't work:
Code: (javascript) [Select]
Debug.log("Can't deserialize: {?}", o.toSource(), LIB_ERROR);


It craps out with a "not callable" error because Duktape doesn't support .toSource().  Apparently that's a Spidermonkey-specific thing.  Duktape doesn't track source code for functions at all, even calling .toString() on a function just gives you an empty function body.

The debugger is such an awesome thing to have by the way, normally with Sphere I would have to drill down into the project tree to find the file (annoying when there are multiple levels of subfolders), then scroll to the correct line, instead I just let the debugger open the file and highlight the line for me. :)
Title: Re: minisphere 2.0b2 (stable 1.7.11)
Post by: Fat Cerberus on October 24, 2015, 11:55:24 pm
Last-minute debugger feature addition: You can double-click any callstack entry while stopped at a breakpoint to see the values of variables in that function.  Unfortunately you can only Eval in the context of the topmost call, so you can only view primitive values in this case. :(

edit: Scratch that, Eval is supported now as well. :)
Title: Re: minisphere 2.0.0
Post by: Fat Cerberus on October 26, 2015, 09:03:53 pm
minisphere 2.0 is out! ;D

A few planned features got knocked off the checklist, but regardless this is still a very worthy upgrade.  The only thing I didn't get to yet was writing an API reference for Cell, but seeing as that API consists of exactly four functions, it shouldn't be a big issue.  I'll get to it eventually.

The debugger improvements are worth the upgrade alone--you can now view variables at any point in the call chain, not just from the currently executing function.  Plus you get some neat goodies like 16-bit and float support for Audialis, better integration of ArrayBuffers, and of course, the new S2GM format.
Title: Re: minisphere 2.0.0
Post by: N E O on October 28, 2015, 05:10:38 pm
Remind me to try 2.0.0 on my OSX Mavericks this weekend!
Title: Re: minisphere 2.0.0
Post by: Fat Cerberus on October 28, 2015, 09:52:04 pm
Sure, you should be good as far as Allegro goes, I'm still using 5.1.11.  minisphere and Cell both build with no issues under Linux using SCons, so you should be good.  Unfortunately, single-step debugging is Windows-only for the moment because I haven't made a command-line debugger yet, so you need Sphere Studio for that.
Title: Re: minisphere 2.0.1
Post by: Fat Cerberus on October 30, 2015, 01:33:35 am
Just posted minisphere 2.0.1 which fixes a bunch of crashes and adds an API reference for Cell.  The installer has been improved somewhat, too.
Title: Re: minisphere 2.0.1
Post by: Fat Cerberus on October 30, 2015, 07:25:13 pm
New feature in latest Git build: SetScreenSize() API allowing to change screen resolution on the fly.  No more ugly hacks modifying the .sgm and restarting. :P

edit: Sneak peak of what's in the pipeline so far for minisphere 2.1:

Title: Re: minisphere 2.0.1
Post by: Fat Cerberus on November 01, 2015, 01:53:20 am
Regarding the checked extensions, you have to use an S2GM manifest first of all, and add a minimumPlatform object to it, like this:

Code: (javascript) [Select]

install(s2gm({
name: "Spectacles: Bruce's Story",
author: "Fat Cerberus",
summary: "Follow Scott Starcross in his quest to stop the Primus.",
resolution: '320x240',
script: 'scripts/main.js',

minimumPlatform: {
recommend: "minisphere 2.1 or later",
apiVersion: 2.0,
extensions: [
"sphere-legacy-api",
"sphere-obj-constructors",
"sphere-obj-props",
"sphere-map-engine",
"sphere-rng-object",
"sphere-s2gm",
"sphere-spherefs",
]
},
}));


The engine will then check for required extensions when loading the game, before any JS code is even executed.  This is better than leaving the requirements check completely up to script, as RequireScript()'d scripts may actually cause a runtime error before the check can be done if they use any extended functionality.
Title: Re: minisphere 2.1.0
Post by: Fat Cerberus on November 01, 2015, 10:39:50 am
In what must be the fastest turnaround for a release in minisphere history, even by my standards, minisphere 2.1 is now available for download! ;D

Now I think I'll take a much-needed break from my tireless work on the engine to work on Spectacles instead... It is, after all, the whole reason I started working on minisphere on the first place. :P
Title: Re: minisphere 2.1.0
Post by: FBnil on November 17, 2015, 09:35:03 pm
You say that... but I still get updates on the git pull.
Pretty impressive so far.
Found some time to play Blackfoot. Very nice block-pushing puzzle game.
Title: Re: minisphere 2.1.0
Post by: Fat Cerberus on November 17, 2015, 10:11:38 pm

You say that... but I still get updates on the git pull.
Pretty impressive so far.
Found some time to play Blackfoot. Very nice block-pushing puzzle game.


Yeah, mostly it's just fixing little annoyances I run into while working on Specs.
Title: Re: minisphere 2.1.1
Post by: Fat Cerberus on November 25, 2015, 02:31:40 am
Quick update (2.1.1): JS errors are now more informative thanks to recent enhancements to Duktape.  Instead of a vague "invalid base value" or "not string", you now get "cannot read property 'foo' of undefined" or "number required, found string 'foobar'" and similar.
Title: Re: minisphere 2.1.1
Post by: N E O on December 02, 2015, 02:13:30 pm
I just pulled the latest source, but I've forgotten how to compile on Mac.  XP

Please refresh me on compilation instructions, thanks!
Title: Re: minisphere 2.1.1
Post by: Fat Cerberus on December 02, 2015, 02:18:42 pm
You need:
Allegro 5.1.11+ (may have to build this yourself, https://github.com/liballeg/allegro5)
libmng
zlib

From the command line, switch to the minisphere source directory and run scons which will build minisphere and Cell in the bin/ subdirectory.
Title: Re: minisphere 2.1.1
Post by: N E O on December 02, 2015, 02:59:24 pm

You need:
Allegro 5.1.11+ (may have to build this yourself, https://github.com/liballeg/allegro5)
libmng
zlib

From the command line, switch to the minisphere source directory and run scons which will build minisphere and Cell in the bin/ subdirectory.


Gave a missing libmng error (pretty much the only library error left after all the previous work I did before), DLed libmng 2.0.3 from Sourceforge, one `./configure; make; sudo make install` later was able to scons minisphere!

Running my two tests worked ok, running Radnen's Cellar Rush gave a LoadSound() error - failed to load some ogg file. I've previously had trouble building Allegro on this Mac with extended sound capabilities (including anything libogg related), so this one's probably on me.

Other than Allegro-specific or me-specific things like the libogg issue, it builds on Mac OS X Mavericks! I'll take a look at Allegro stuff later.

This current building served two purposes: make sure current minisphere builds and runs on Mavericks, and set up a prep environment related to my game dev and my RFC about adding Python (http://forums.spheredev.org/index.php/topic,1320.0.html).
Title: Re: minisphere 2.1.1
Post by: Fat Cerberus on December 02, 2015, 03:21:49 pm
Here's the thing with Allegro: By default it dynamically loads audio codecs such as libogg at runtime, and gracefully degrades if they are missing (this is annoying).  You can static link them if you disable the CMAKE flag WANT_ACODEC_DYNAMIC_LOAD, not sure how feasible that is on OS X though.  Either way, additional dependencies needed for Allegro audio:

dumb
libFLAC
libogg
libvorbis
libvorbisfile

They really need to get rid of that "dynamic link by default" feature, it causes so many headaches.
Title: Re: minisphere 2.1.1
Post by: Fat Cerberus on December 03, 2015, 11:43:58 pm
@NEO: Did you get Allegro to build properly with sound support?
Title: Re: minisphere 2.1.1
Post by: N E O on December 04, 2015, 01:37:46 pm

@NEO: Did you get Allegro to build properly with sound support?


Not yet; for reference, Allegro's cmake is giving me the following output (tl;dr - I'm missing libFLAC, libdumb, libvorbis):

Quote

-- Allowing GCC/Clang to use SSE instructions
-- Could NOT find PkgConfig (missing:  PKG_CONFIG_EXECUTABLE)
S3TC locking disabled. You will not be able to load/save pre-compressed textures with OpenGL.
-- Could NOT find OPENSL (missing:  OPENSL_INCLUDE_DIR OPENSL_LIBRARY)
-- Could NOT find FLAC (missing:  FLAC_INCLUDE_DIR)
WARNING: libFLAC not found or compile test failed, disabling support.
-- Could NOT find DUMB (missing:  DUMB_INCLUDE_DIR DUMB_LIBRARY)
WARNING: libdumb not found or compile test failed, disabling support. <http://dumb.sourceforge.net/>
-- Could NOT find OGG (missing:  OGG_INCLUDE_DIR)
WARNING: libvorbis not found or compile test failed, disabling support.
-- Could NOT find OGG (missing:  OGG_INCLUDE_DIR)
-- Could NOT find OGG (missing:  OGG_INCLUDE_DIR)
WARNING: allegro_video wanted but no supported backend found
-- Not building ex_video
-- Configuring done
CMake Warning (dev):
  Policy CMP0042 is not set: MACOSX_RPATH is enabled by default.  Run "cmake
  --help-policy CMP0042" for policy details.  Use the cmake_policy command to
  set the policy and suppress this warning.

  MACOSX_RPATH is not specified for the following targets:

   allegro
   allegro_acodec
   allegro_audio
   allegro_color
   allegro_dialog
   allegro_font
   allegro_image
   allegro_main
   allegro_memfile
   allegro_physfs
   allegro_primitives
   allegro_ttf

This warning is for project developers.  Use -Wno-dev to suppress it.

-- Generating done
-- Build files have been written to: /Users/lux/dev/allegro5/Build
Title: Re: minisphere 2.1.1
Post by: Fat Cerberus on December 05, 2015, 12:35:28 am
You think this is bad, you should have seen me try to build Allegro on Windows when I first upgraded to 5.1.  Even with CMake it's a nightmare, sometimes it doesn't find libraries that are actually present, or they'll fail a compile test for no reason at all (which CMake then "remembers" and won't try again until you blow away the entire build cache and start over).  This is why I'm still using Allegro 5.1.11 even though 5.1.12 has some new stuff that would actually be useful to have (such as cross-platform clipboard support).

Allegro is a great library, but their build process needs a lot of work.
Title: Re: minisphere 2.1.1
Post by: Flying Jester on December 05, 2015, 03:53:29 am
Even with CMake it's a nightmare


CMake is just always a bit of a nightmare. This is a pretty normal experience with it, especially on Windows.
Title: Re: minisphere 2.1.1
Post by: Fat Cerberus on December 08, 2015, 01:17:10 pm
I got minisphere to compile with Chipmunk2D linked in.  So minisphere 2.2 will definitely have some kind of physics support, once I figure out a good API structure.

Originally I was going to try to copy the particle engine from Sphere 1.6, but I'm thinking now that a more general-purpose physics engine will be better in the long run.
Title: Re: minisphere 2.1.1
Post by: Fat Cerberus on December 11, 2015, 12:36:38 pm

Even with CMake it's a nightmare


CMake is just always a bit of a nightmare. This is a pretty normal experience with it, especially on Windows.


In large part this seems to be caused by CMake's braindead caching setup: When a variable is set, its value is stored in the CMakeCache and future accesses will always just read from the cache.  There doesn't appear to be any kind of conditional logic where the cached value can be ignored when something related to it changes.
Title: Re: minisphere 2.1.3
Post by: Fat Cerberus on December 15, 2015, 02:41:23 am
Just implemented: Error copying.  Finally you can press Ctrl+C on the error screen to copy error text (and filename/line number) directly to the clipboard.  This has been a long time coming.
Title: Re: minisphere 2.1.3
Post by: Radnen on December 15, 2015, 02:47:12 am

Just implemented: Error copying.  Finally you can press Ctrl+C on the error screen to copy error text (and filename/line number) directly to the clipboard.  This has been a long time coming.


Ok, that's it. I'm done with Sphere 1.5/1.6. That was the secret. I wanted you to do that, but never said, and now that you did it, bye bye Sphere 1.5/1.6. :)
Title: Re: minisphere 2.1.3
Post by: Fat Cerberus on December 15, 2015, 03:20:52 am

Ok, that's it. I'm done with Sphere 1.5/1.6. That was the secret. I wanted you to do that, but never said, and now that you did it, bye bye Sphere 1.5/1.6. :)


Hehe.  I wanted to do this a while ago, but I was too lazy to research the clipboard APIs for the different platforms.  As of Allegro 5.1.12, basic clipboard functions are built-in, so I took advantage of it.

2.1.3 doesn't include the feature, but it's in the git repository anyway (Windows build requires VC++2015).  If I release a 2.1.4 I'll backport it, otherwise it'll be in v2.2 along with some sort of physics support.  I haven't quite worked out a good API for that yet.
Title: Re: minisphere 2.1.4
Post by: Fat Cerberus on December 16, 2015, 12:22:39 am
Okay, Radnen, you lucked out, I had to fix a map engine bug related to GrabImage() (actually a regression due to a careless recent change to the map update loop) so I released a hotfix.  Said hotfix happens to include the error message copying, so now you can download minisphere 2.1.4 and ditch Sphere 1.x if you want. ;)

As for physics, the API is going to need some more thought before I forge ahead with it, so I'm postponing it to minisphere 3.0.  Instead I think I'll dedicate minisphere 2.2 to improving Cell.  What was released of it with minisphere 2.0 is little more than a prototype and I'd like to make it a more invaluable part of the Sphere toolchain.
Title: Re: minisphere 2.1.4
Post by: casiotone on December 16, 2015, 11:20:53 am
Having an issue building this again on OS X - I've installed Allegro 5.0.11 through homebrew and I'm getting this when running scons:

Code: [Select]
scons: Reading SConscript files ...
scons: done reading SConscript files.
scons: Building targets ...
gcc -o obj/msphere/main.o -c -DDUK_OPT_HAVE_CUSTOM_H obj/msphere/main.c
obj/msphere/main.c:183:44: error: use of undeclared identifier 'ALLEGRO_PROGRAMMABLE_PIPELINE'
        al_set_new_display_flags(ALLEGRO_OPENGL | ALLEGRO_PROGRAMMABLE_PIPELINE);
                                                  ^
obj/msphere/main.c:624:5: warning: implicit declaration of function 'al_set_clipboard_text' is invalid in C99
      [-Wimplicit-function-declaration]
                                al_set_clipboard_text(g_display, msg);
                                ^
obj/msphere/main.c:779:13: warning: using the result of an assignment as a condition without parentheses [-Wparentheses]
        while (fse = al_read_directory(engine_dir)) {
               ~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
obj/msphere/main.c:779:13: note: place parentheses around the assignment to silence this warning
        while (fse = al_read_directory(engine_dir)) {
                   ^
               (                                  )
obj/msphere/main.c:779:13: note: use '==' to turn this assignment into an equality comparison
        while (fse = al_read_directory(engine_dir)) {
                   ^
                   ==
2 warnings and 1 error generated.
scons: *** [obj/msphere/main.o] Error 1
scons: building terminated because of errors.


Any ideas?
Title: Re: minisphere 2.1.4
Post by: Fat Cerberus on December 16, 2015, 11:45:36 am
Yeah, you need Allegro 5.1 "unstable" (i.e. the API is unstable, it's actually quite reliable).  Not sure if that's available through homebrew; you may need to build Allegro yourself:
http://github.com/liballeg/allegro5
Title: Re: minisphere 2.1.4
Post by: casiotone on December 17, 2015, 07:46:36 am
Thanks, managed to build Allegro. Now having a problem with libmng - I can't get that to build on OS X at all. I have no plans to actually use MNG, is there any chance you could add an option to compile without MNG support?
Title: Re: minisphere 2.1.4
Post by: Fat Cerberus on December 17, 2015, 10:17:35 am
You're the second person to request that, I'll see what I can do.  For now you can probably disable it by:
* Remove 'animation.c' from msphere_sources in the SConscript
* Remove 'mng' from msphere_libs in the SConscript
* Comment out the call to init_animation_api() in api.c

There's been talk of Allegro adding optional MNG support to the library; if that happens I can do away with my own libmng dependency.  Oh, that reminds me, watch out with Allegro's audio support; if you didn't set WANT_ACODEC_DYNAMIC_LOAD=0 when using Allegro's CMake, then it attempts to dynamically load libvorbis, etc. at runtime and fails silently if it can't, so if minisphere fails to load any sound files--it's most likely not my fault! ;)
Title: Re: minisphere 2.1.4
Post by: Fat Cerberus on December 17, 2015, 10:22:03 pm
By the way, casiotone: I just found out you didn't have to compile Allegro from source after all.  5.1.12 was available on homebrew after all, the command is:
brew install --devel allegro

@NEO: You might want to take note of the above as well. :)
Title: Re: minisphere 2.1.4
Post by: N E O on December 17, 2015, 11:07:23 pm

By the way, casiotone: I just found out you didn't have to compile Allegro from source after all.  5.1.12 was available on homebrew after all, the command is:
brew install --devel allegro

@NEO: You might want to take note of the above as well. :)


Two errors:

Code: [Select]

airlux:magento lux$ brew install --devel allegro
Error: You must `brew link libogg libvorbis cmake freetype flac' before allegro can be installed


...then...

Code: [Select]

airlux:magento lux$ brew link libogg libvorbis cmake freetype flac
Linking /usr/local/Cellar/libogg/1.3.2...
Error: Could not symlink lib/libogg.0.dylib
Target /usr/local/lib/libogg.0.dylib
already exists. You may want to remove it:
  rm '/usr/local/lib/libogg.0.dylib'

To force the link and overwrite all conflicting files:
  brew link --overwrite libogg

To list all files that would be deleted:
  brew link --overwrite --dry-run libogg


Give me a bit of time to eventually focus on doing this and I'll let y'all know results.
Title: Re: minisphere 2.1.4
Post by: casiotone on December 18, 2015, 01:59:23 pm

You're the second person to request that, I'll see what I can do.  For now you can probably disable it by:
* Remove 'animation.c' from msphere_sources in the SConscript
* Remove 'mng' from msphere_libs in the SConscript
* Comment out the call to init_animation_api() in api.c

There's been talk of Allegro adding optional MNG support to the library; if that happens I can do away with my own libmng dependency.  Oh, that reminds me, watch out with Allegro's audio support; if you didn't set WANT_ACODEC_DYNAMIC_LOAD=0 when using Allegro's CMake, then it attempts to dynamically load libvorbis, etc. at runtime and fails silently if it can't, so if minisphere fails to load any sound files--it's most likely not my fault! ;)


Works like a charm, thanks again.
Title: Re: minisphere 2.1.5
Post by: Fat Cerberus on December 19, 2015, 01:08:56 am
minisphere 2.1.5 fixes corruption at the end of a WAV stream (note: this was an Allegro bug) and adds an option to the GDK plugin to Test Game in windowed mode.  The default is still fullscreen, of course.
Title: Re: minisphere 2.1.6
Post by: Fat Cerberus on December 24, 2015, 01:38:13 pm
So I've been thinking on and off about the direction I want to go with in minisphere 3.0, but I've been wracking my brain because pretty much all of what I set out to implement has been implemented as of minisphere 2.1.  I haven't even been able to muster a clear plan for a v2.2 (which will likely end up just being some Cell enhancements), let alone a major upgrade.  Which really only leaves one more hurdle: plugins.  The engine already has most of the framework in place for this (the extension system), most of the work would be coming up with an API.
Title: Re: minisphere 2.1.6
Post by: FBnil on December 25, 2015, 12:37:36 pm
http://www.drdobbs.com/cpp/building-your-own-plugin-framework-part/204202899

I like DrDobbs magazine for the conceptual things it contains. Just relax, sit down in non-programming mode, and read it. Around page 3/4 it goes into code... maybe it gives you an idea on how to start the API.
Title: Re: minisphere 2.1.6
Post by: DaVince on December 29, 2015, 07:51:22 pm
Hey, so this is just my selfishness speaking, but how feasible would a software renderer mode be? :P I'm developing something while temporarily on quite a crappy computer (Supports up to OpenGL 1.3...) so I obviously stumbled against the issue of not being able to run minisphere.
Title: Re: minisphere 2.1.6
Post by: Fat Cerberus on December 29, 2015, 07:54:21 pm
Hm, in theory it's possible to initialize Allegro in D3D mode, which would raise compatibility.  The downside is that shader support would be disabled, but it would allow the rest of the functionality (including base Galileo) to work.  I'll see about making that change in minisphere 2.2.
Title: Re: minisphere 2.1.6
Post by: Flying Jester on January 01, 2016, 07:30:54 pm
I've read that Dr Dobb's article, and I'm not really a fan of how they do it.

I did about the simplest thing in TurboSphere. The API looked like this:

Code: (C) [Select]

const char *Init(JSContext *ctx); // Returns the plugin name.

int GetNumFunctions(JSContext *ctx);
const char *GetFunctionName(JSContext *ctx, int i);
JSNative GetFunction(JSContext *ctx, int i);

int GetNumVariables(JSContext *ctx);
const char *GetVariableName(JSContext *ctx, int i);
void GetVariable(JSContext *ctx, int i, JSMutableHandle out); // It's much better to use an out-parameter for handles with SpiderMonkey.

void Close(); // Closes the plugin.


I'm still using a variant on that in UltraSphere/Turbo. I guess why complicate things, I never found I needed anything more than this. With how minisphere handles extensions, you could also add a GetExtension/GetNumExtensions component.

That's just my two cents from my own experiments.
Title: Re: minisphere 2.1.6
Post by: Fat Cerberus on January 01, 2016, 09:58:37 pm

I've read that Dr Dobb's article, and I'm not really a fan of how they do it.


I have to agree.  I looked over it and the very OO method they provided just seemed overcomplicated for most purposes.  That said, I did use a similar setup when I overhauled Sphere Studio's plugin API, but at least there it's justified since 1) There are a bunch of different plugin roles (debugger, editor, engine runner, and so on) but I needed to be able to have one plugin implement multiple roles while still having a unified interface, and 2) It's C#, so I don't have to worry about name mangling differences, etc. like I would with C++.
Title: Re: minisphere 2.1.6
Post by: DaVince on January 03, 2016, 02:02:38 pm
minisphere now runs on my ancient OpenGL 1.3-only system. Thanks a lot for the support, Lord English. :)
Title: Re: minisphere 2.1.6
Post by: Fat Cerberus on January 07, 2016, 02:36:58 am
Having just started work on minisphere 3.0, a question occurs to me: Would anyone be opposed if, for the 3.0 Windows release, I provided only a 64-bit build?  The x86 build seems pretty redundant nowadays, I personally haven't encountered a 32-bit version of Windows in years.
Title: Re: minisphere 2.1.6
Post by: FBnil on January 07, 2016, 05:48:17 pm
On older systems, there are many other game makers... go for gold. When the time comes, compile for 128-bit.
Before you tag and go to 3, can you take a tiny look at Linux? My latest succesful build was Nov 29. And

Code: [Select]
obj/msphere/main.o: In function `main':
main.c:(.text+0x3ce): undefined reference to `al_set_new_window_title'
obj/msphere/main.o: In function `on_duk_fatal':
main.c:(.text+0x1e32): undefined reference to `al_set_clipboard_text'
collect2: error: ld returned 1 exit status
scons: *** [obj/msphere/msphere] Error 1
scons: building terminated because of errors.
Title: Re: minisphere 2.1.6
Post by: Fat Cerberus on January 07, 2016, 05:55:01 pm
You need Allegro 5.1.12 for the current builds.  If you're using Ubuntu there are PPA packages available, otherwise you'll have to compile it yourself.

Basically whenever you get undefined references to a function starting with al_, you need to update Allegro.
Title: Re: minisphere 2.1.6
Post by: DaVince on January 08, 2016, 07:50:10 pm
Is there a way to make the build system spew out that bit of information?

Also, I don't mind you dropping the 32-bit Windows version too much. Poor Windows XP users though... :P
Title: Re: minisphere 2.1.6
Post by: Flying Jester on January 08, 2016, 07:51:47 pm
It would be simple in SCons using `Configure` to check for a certain library and then function in it, and then print out such a warning.
Title: Re: minisphere 2.1.6
Post by: Fat Cerberus on January 08, 2016, 07:58:14 pm
Actually does minisphere even run on WinXP?  Defiant had issues a while back, I think I fixed it but I don't have an XP machine to test on.

I'm going to work on improving the documentation for minisphere 3.0.  Developer documentation is still pretty scant right now because I've been lazy.  I need to get off my duff for 3.0. :P
Title: Re: minisphere 2.1.6
Post by: FBnil on January 09, 2016, 04:45:53 pm
git tag just shows 5.1.11 (from git clone git://git.code.sf.net/p/alleg/allegro) will wait for 5.1.12 because my HEAD is not compiling at the moment.
Title: Re: minisphere 2.1.6
Post by: Fat Cerberus on January 09, 2016, 04:50:43 pm
Try cloning from https://github.com/liballeg/allegro5 instead.  They switched to GitHub recently so I don't think the SourceForge repo is maintained anymore.
Title: Re: minisphere 2.1.6
Post by: FBnil on January 09, 2016, 05:06:34 pm
Wow, quick, yes git clone https://github.com/liballeg/allegro5.git
has a 5.1.12.0 tag...

Sorry for asking, in another thread I stated:
Quote
Allegro "make" only plays nice if I disable FFMPEG and CG

.... but I can not remember how I did that (and it is not in my howto.txt I always write what I do...)
Title: Re: minisphere 2.1.6
Post by: Fat Cerberus on January 09, 2016, 05:52:10 pm
Did you try Allegro's cmake?
Title: Re: minisphere 2.1.6
Post by: FBnil on January 09, 2016, 07:22:44 pm
When running for the second time I get

Code: [Select]
-- Allowing GCC/Clang to use SSE instructions
S3TC locking disabled. You will not be able to load/save pre-compressed textures with OpenGL.
-- Could NOT find OPENSL (missing:  OPENSL_INCLUDE_DIR OPENSL_LIBRARY)
-- Configuring done
-- Generating done
-- Build files have been written to: /home/fbnil/MiniSphere/allegro5/Build

But I do not remember requiring that last time. Do you mean editing AllegroFindFFMPEG.cmake there remove FFMPEG... and... yes... now it compiles, and I can make install. hmm... did not recall this. Same error in minisphere though. Will check tomorrow, last time, I had to purge the previous Allegro version before it started working (too late now, need Zzzz)
Title: Re: minisphere 2.1.6
Post by: Fat Cerberus on January 09, 2016, 07:27:03 pm
If Allegro cmake can't find FFMPEG it should just disable the video component (or at least that's how it's supposed to work).  This is fine since minisphere doesn't use that part.  You should be in the clear.
Title: Re: minisphere 2.1.6
Post by: Fat Cerberus on January 09, 2016, 07:38:26 pm
To everyone else: I'm in the process of writing a full install guide for minisphere.  This should hopefully alleviate some of the issues seen in the past for people building minisphere on non-Windows platforms.
Title: Re: minisphere 2.1.6
Post by: DaVince on January 09, 2016, 08:27:37 pm

Actually does minisphere even run on WinXP?  Defiant had issues a while back, I think I fixed it but I don't have an XP machine to test on.


Well, I could probably test this soon... Just need to shrink my Linux partition on this old PC and install XP on it. Depending on the last version that works, you could still offer that as the last 32-bit version download for those who really are stuck on 32-bit Windows. :)
Title: Re: minisphere 2.1.6
Post by: Defiant on January 10, 2016, 12:06:25 am

Actually does minisphere even run on WinXP?  Defiant had issues a while back, I think I fixed it but I don't have an XP machine to test on.


I haven't got it to work yet on my XP machine. Doesn't crash, just loads up a dialog and then exits, can't see the error before it disappears.
Title: Re: minisphere 2.1.6
Post by: Fat Cerberus on January 10, 2016, 12:09:23 am
Huh, that's weird.  Perhaps I'll try setting up a VM with XP to try to diagnose it.
Title: Re: minisphere 2.1.6
Post by: Fat Cerberus on January 10, 2016, 03:23:56 am
For SSJ, to make it as efficient as possible to debug on the command-line, I think I want to go with 1- or 2-character commands, for example "s" to step, "st" for a stack trace, "v" to list local variables, and so on.  No prefixing commands with a dot or ! as that just slows down command entry.

I'm not sure yet how to handle locating sources for a game.  Cell can optionally include a source map if needed when building a game (this is very useful as it allows SPKs to be debugged!), but even that uses relative paths.  In Sphere Studio it's trivial to just resolve the relative path against the current project, but SSJ will need a command-line option or something to provide the location of the sources.

What features would you want to see in the debugger?  Keep in mind some things might not be possible until Duktape implements them in its debug API, but I'll make sure to note any requests regardless. :)
Title: Re: minisphere 2.1.6
Post by: FBnil on January 10, 2016, 08:56:03 am
I would like a break command in the code to set a breakpoint. If the break command can have an identifier, to know which one breaked....
Title: Re: minisphere 2.1.6
Post by: Fat Cerberus on January 11, 2016, 02:57:02 am
I don't know if anyone remembers the --explode switch you could pass to Cell to get an easter egg (specifically, a quote from the program's namesake), but I'm starting a tradition with SSJ where all the GDK tools will have this little secret.

For point of reference, sample output from Cell:
Code: [Select]

C:\Users\fatce>cell --explode
Cell seems to be going through some sort of transformation...
He's pumping himself up like a balloon!

    Cell says:
    "If only you'd finished me off a little bit sooner..."


And now, from SSJ:
Code: [Select]

C:\Users\fatce>ssj --explode
Release it, release everything! Remember all the pain he's caused, the people
he's hurt--now MAKE THAT YOUR POWER!

    Cell says:
    "Would you stop interfering!?"


;D

Let nobody say I don't have lots of fun while programming these tools!
Title: Re: minisphere 2.1.6
Post by: Fat Cerberus on January 11, 2016, 09:13:53 am
Also see this:
https://github.com/fatcerberus/minisphere/issues/56

:P
Title: Re: minisphere 2.1.6
Post by: Flying Jester on January 11, 2016, 11:24:46 pm


Actually does minisphere even run on WinXP?  Defiant had issues a while back, I think I fixed it but I don't have an XP machine to test on.


I haven't got it to work yet on my XP machine. Doesn't crash, just loads up a dialog and then exits, can't see the error before it disappears.


Try running it in a cmd.exe console? That's usually the thing to do when errors pop up too fast to read.
Title: Re: minisphere 2.1.6
Post by: Defiant on January 13, 2016, 11:13:52 pm
So it crashes when it tries to create the window. I copied the data it contains in the technical stuff, let me know if anything else would help diagnose this.
Title: Re: minisphere 2.1.6
Post by: Fat Cerberus on January 13, 2016, 11:48:15 pm
I see you're using minisphere 1.7.4.  I was going to suggest upgrading to 2.1.6, but I think I know the cause of the bug: Your machine might not support OpenGL shaders.  minisphere currently requires shader support to run.  In the latest builds I added a fallback for that, but I didn't backport the fix to 2.1 because I figured it wasn't a practical concern.  Seems I was wrong. ;)

I'll try running minisphere in my XP VM to verify my theory.

edit: Hm.  The last line printed out by minisphere 2.1.6 on my Hyper-V WinXP SP3 VM is:
Code: [Select]

Looking for a game to launch


Seemingly no crash, though.  Really weird.  I'll review the code for the "looking for a game" to see if I can find anything.
Title: Re: minisphere 2.1.6
Post by: Fat Cerberus on January 14, 2016, 12:27:35 am
Interesting oddity I found: Cell doesn't work properly under XP either.  I tried compiling Spectacles with it, and was told "no Cellscript.js in input directory", which is patently false.  So it seems this might be file system-related.  I'll keep digging.
Title: Re: minisphere 2.1.6
Post by: Defiant on January 14, 2016, 10:51:58 pm
Sorry, I should clarify that I can load a game, doesn't matter which one, but it fails to load the window when launching the game, sorry for the confusion.
Title: Re: minisphere 2.1.6
Post by: Fat Cerberus on January 14, 2016, 11:04:48 pm
So it successfully runs the startup game?  That's quite strange then.
Title: Re: minisphere 2.1.6
Post by: Defiant on January 15, 2016, 06:13:20 pm
It successfully loads minisphere. It allows me to select a game, but then crashes there after. It doesn't load the game.
Title: Re: minisphere 2.1.6
Post by: Fat Cerberus on January 17, 2016, 08:24:33 pm
Good news everyone: I finally got Ubuntu 14.04 successfully installed on my laptop to dual boot with Win10.  What this means is that I can actively support the engine code on both Windows and Linux now (or Ubuntu, at least).  Now if only I had a Mac, we'd have the trifecta...

If I could get some official Ubuntu packages going that'd be great for exposure too, I think.
Title: Re: minisphere 2.1.6
Post by: FBnil on January 18, 2016, 04:56:14 pm
Yes, that would certainly help a lot. You have time until Allegro 5.x hits Ubuntu (at least, your unstable version with the code you need to get it compiled/running).
Also: Documentation documentation documentation.
From using it as node.js (sphere was a crappy node.js, but I could serve webpages)... hmm.. maybe not this one...
to debugger (good work getting it working!)
to using it like image magick (I have some turtle code I think lost it) read png, add text, write to jpg things. Read file, parse data, create graph...
to well... a game making software.
Bundle it with some small default examples (from pianola to slideshow, an rpg, a sidescroller, things like that)

I still need to get into the code to add the things I miss (like changing the spriteset directions labels to "swap" graphics) and test out if I finally can access the spriteset image to mix it (overlay different hair (processed through a color matrix) and such to generate spritesets on the fly).

oh boy oh boy... very good news...
Title: Re: minisphere 2.1.6
Post by: N E O on January 19, 2016, 02:18:44 am

Now if only I had a Mac, we'd have the trifecta...


If you personally invest money in a Mac, I highly recommend getting into iOS app dev at that point, since IMO that's the only thing a Mac can do properly that no other OS can come close to (basically the iOS equivalent of the C#+Visual Studio vs. Mono+MonoDevelop and/or maybe the wine vs. Windows debate).
Title: Re: minisphere 2.1.6
Post by: Fat Cerberus on January 19, 2016, 03:30:15 am
I have to say I'm not liking Linux as a development environment.  It has all kinds of neat tools like valgrind (note: minisphere, Cell and SSJ all leak like a sieve :P), and of course the package system makes it ridiculously quick and simple to install new stuff, but for everyday code maintenance I'd much rather have Visual Studio, bloat and all.

Doing everything from the terminal and using a text editor to edit source files feels like a massive downgrade after being spoiled by Intellisense.  I know there are IDEs that work on Linux, but... eh.  I think I'll just stick to Windows for primary development. ;)
Title: Re: minisphere 2.1.6
Post by: Flying Jester on January 19, 2016, 04:31:37 am
Not having Intellisense can be a good thing, too. It made me realize that if I can never remember my own API, it can be a sign it is not designed well.
Title: Re: minisphere 2.1.6
Post by: Fat Cerberus on January 19, 2016, 09:40:43 am
It's less that I can't remember the API so much as I strive for utmost readability, leading to descriptive struct member names like ignore_all_persons.  Imagine how much time it saves when Intellisense figures out, nine times out of ten, exactly which variable I want after typing only one character (and sometimes not even that!).  Then just press Tab to accept.  It's a tremendous time-saver.  I could survive without it, but code readability would likely suffer for it, as I would be too tempted to abbreviate all my variable and function names.
Title: Re: minisphere 2.1.6
Post by: FBnil on January 19, 2016, 02:13:17 pm
Well there are some IDE's that have some of the VC properties:

https://en.wikipedia.org/wiki/Code%3A%3ABlocks
https://en.wikipedia.org/wiki/Eclipse_%28software%29
https://en.wikipedia.org/wiki/Qt_Creator

I found Eclipse slow (java based) although some of my colleagues praise it so much its religous, Codeblocks very fast (runs on the OpenPandora) and complete (but not the stable release, that one is 2 years old, you really want the rc1 from a month ago). And Not sure what happened to Qt Creator, it seems like gone, but was very good a few years ago.

I am curious about http://anjuta.org/ so I'm going to give it a spin.
Never tried this one, but it looks also like something you would want to try:
https://www.kdevelop.org/sites/kdevelop.org/files/photos/Kdevelop_cpp_codetooltip.png

I actually use Notepad++ under wine. Gives me all I need.
Title: Re: minisphere 2.1.6
Post by: Radnen on January 19, 2016, 04:09:58 pm

I have to say I'm not liking Linux as a development environment.  It has all kinds of neat tools like valgrind (note: minisphere, Cell and SSJ all leak like a sieve :P), and of course the package system makes it ridiculously quick and simple to install new stuff, but for everyday code maintenance I'd much rather have Visual Studio, bloat and all.

Doing everything from the terminal and using a text editor to edit source files feels like a massive downgrade after being spoiled by Intellisense.  I know there are IDEs that work on Linux, but... eh.  I think I'll just stick to Windows for primary development. ;)


Use MonoDevelop. It's really good and had decent intellisense last I used it. Plus, apps you do happen to make on it are Mono and therefore cross-platform. I might just restart my Mono Sphere Editor one of these days...
Title: Re: minisphere 2.1.6
Post by: Fat Cerberus on January 19, 2016, 05:17:36 pm
I've been trying to get Sphere Studio to run under Mono, but no such luck.  Before the huge refactoring I did a few months back, it used to throw an exception related to the Weifen Luo controls.  Now it doesn't produce any output at all, and terminates immediately.  Not sure what's causing it and I don't know how to get debugging information out of Mono.
Title: Re: minisphere 2.1.6
Post by: Fat Cerberus on January 20, 2016, 12:41:52 am
Follow-up: I managed to pin down the Wine lockup.  It locks up whenever it tries to open a document tab.  Wine, like Mono, really seems to hate the Weifen Luo dock panel. :-\
Title: Re: minisphere 2.1.6
Post by: N E O on January 20, 2016, 01:49:10 pm

Follow-up: I managed to pin down the Wine lockup.  It locks up whenever it tries to open a document tab.  Wine, like Mono, really seems to hate the Weifen Luo dock panel. :-\


Do the Weifen Luo widgets need .NET >=4.0 or use some obscure, undocumented Win32 |API?
Title: Re: minisphere 2.1.6
Post by: Fat Cerberus on January 20, 2016, 01:57:05 pm
Sphere Studio itself requires .NET 4.5.  It's impossible to backport now since it uses async/await pretty extensively, especially in the debugger code.  But that's not the issue I don't think--everything else seems to work fine under Wine (even things I didn't expect to work, like running Cell and redirecting it's stdout to the IDE), until you go to open a file.  Then it freezes.

I think the problem is that the Weifen Luo controls abuse P/Invoke for calling into native win32 APIs, to support things like interprocess drag-and-drop.  Wine's emulation of the necessary APIs must not be perfect and so it causes deadlocks or something.

The weird thing is that the start page shows up just fine.  It's quite baffling.
Title: Re: minisphere 2.1.6
Post by: Fat Cerberus on January 24, 2016, 02:59:05 am
Here's a sneak peek at SSJ, the new console debugger that will come with minisphere GDK 3.0.  It has fancy colors and everything! ;D  As a convenience, SSJ will launch minisphere for you, meaning all you have to do to start a fresh debugging session is run ssj <filename>.  This is especially convenient on Linux, which previously required using two terminals, one to run msphere --debug <filename> and the other for the debugger.

Anyway, check out the screenshot, it's looking pretty slick so far I think.
Title: Re: minisphere 2.1.6
Post by: Flying Jester on January 24, 2016, 02:41:05 pm
This is especially convenient on Linux, which previously required using two terminals, one to run msphere --debug <filename> and the other for the debugger.


Or you could just run minisphere with a `&` after it, to not make it blocking :P
Title: Re: minisphere 2.1.6
Post by: Fat Cerberus on January 24, 2016, 02:48:36 pm
I did find that out, although in my attempts at that the two programs would fight over the same terminal, unless of course you redirect the engine's stdout elsewhere (which is exactly what SSJ does).  In any case this matches how GDB and similar debuggers work, where you pass the program to be debugged as an argument to the debugger.

Basically, in Windows if you do start whatever, it opens a new terminal (console) for the program, I couldn't find a way to do the same in Linux.
Title: Re: minisphere 2.1.6
Post by: Flying Jester on January 24, 2016, 04:23:39 pm
Traditionally, you would explicitly start another terminal (usually `xterm`, which a lot of distros symlink to be their DE's terminal rather than the real xterm), which works similarly to `start` in Windows. If you pass an argument, that is executed by the terminal, which will exit once that command returns.
Title: Re: minisphere 2.1.6
Post by: FBnil on January 25, 2016, 04:45:08 pm
# You can run 2 things like this:
$ xterm -title "This is a terminal1" -r -selbg blue -e "/script/or/program/run"  &
$ xterm -title "This is a terminal2" -e "vi" &
# The wait statement waits for all the child processes...
$ wait
# now do some cleanup...
echo "cleaning up"...
Title: Re: minisphere 2.1.6
Post by: Fat Cerberus on January 26, 2016, 02:14:24 am
SSJ commands implemented so far (names subject to change as development progresses):

"go" - Resumes execution.
"step" - Step over, same as "next" in GDB
"in" - Step in
"out" - Step out
"trace" - Prints a numbered stack trace, 0 is the function currently executing, counts up from there
"var" - Prints local variables for the current function.  Doesn't yet show values.
"eval" - Runs arbitrary JS code in the current context, can also be used to examine variables ("eval someVar")
"quit" - Detaches the debugger and terminates

No breakpoint support yet, and the Step commands don't show any source code for context, but these all work otherwise, so I'm off to a good start.  Oh, and it also reports uncaught exceptions.
Title: Re: minisphere 2.1.6
Post by: Flying Jester on January 26, 2016, 10:54:45 am
It might be nice to have 'continue' as a synonym for 'go' and 'print' for 'eval', since that would match lldb and gdb.

Can you just use the first letter of the command to perform it? That's a really nice feature of most debuggers.
Title: Re: minisphere 2.1.6
Post by: Fat Cerberus on January 26, 2016, 11:40:34 am
Right now, single letters aren't accepted as valid commands, but I do want to implement that.  It gets tedious typing 'step' (or selecting it from history) over and over again. :)

For the most part I take inspiration from gdb for command naming, but a few of its naming decisions baffle me.  For example in gdb `step` is actually Step In, while Step Over is mapped to `next`.  This is weird to me, since it's been my experience that the vast majority of stepping done is Step Over.  In most cases with well-placed breakpoints you don't need to step into functions unless the code is really tangled. (the reason this matters is that it's quite a bit faster to type 's'+enter than 'n'+enter)

I will most likely rename `trace` to `backtrace`.  Even though the term is odd to me, it matches gdb and lldb naming and is an accepted term for a call stack listing. As for `print`, I was considering that initially, but hesitated: `eval` can execute arbitrary code with any number of side effects.  Someone using a command called `print` may not expect that.  That said, I'll probably do it anyway though for parity with gdb.

What's your opinion of the command line (see screenshot posted earlier)?  gdb's prompt is just "(gdb)", likewise for lldb, whereas I include the source file and line number.  I'm undecided on whether this is legitimately useful or just adds noise.
Title: Re: minisphere 2.1.6
Post by: Flying Jester on January 26, 2016, 10:49:30 pm
I think the debugger having its own prompt name makes more sense when you have control of stdin, since many programs (in Unix at least) provide line-editing like features.

`print` in lldb does perform JITing in expressions via llvm.

In general if it were me, I'd take more inspiration from gdb than lldb. The latter tends to be much more annoying to use, since it's extremely verbose. The reasoning is that it's more orthogonal and descriptive, but in reality it just makes everything more work.

Most of the unusual names in gdb were just to give each command a unique but initial letter synonym. If you always call stack traces 'backtraces', you will always remember they are `bt` or just `b`.
Title: Re: minisphere 2.1.6
Post by: Fat Cerberus on January 27, 2016, 03:18:24 am
I implemented stack walking, yay!  You can now enter `frame <x>` to switch to a different function in the stack.  Future requests to print variables, run code, etc. will then use that scope.

This is actually coming together pretty quickly!  I do still need to implement source code listing and most importantly, breakpoints, and that should put SSJ pretty close to feature parity with the Sphere Studio plugin.  Heck, I might even be able to rewrite that plugin to use SSJ under the hood, to avoid having to maintain two debugger codebases. ;D
Title: Re: minisphere 2.1.6
Post by: FBnil on January 27, 2016, 03:33:01 pm
.. nice...

I managed to finally compile minisphere, but only after using http://download.gna.org/allegro/allegro-unstable/5.1.13/  (but I also removed all allegro libs from my debian, breaking allegro games, then make installed the new version)
I noticed that --game is not an option anymore, and I couldn't see why in the git log's, so I assume it is a bug?

Code: [Select]
fbnil@convoy$ strings msphere.20150610 |grep ^-- |sort >a ; strings msphere|grep ^-- |sort >b;diff a b
0a1
> --debug
3d3
< --game
6c6,7
< --windowed
---
> --version
> --window

fbnil@convoy$ comm a b
        --debug
                --frameskip
                --fullscreen
--game
                --log-level
                --no-throttle
        --version
        --window
--windowed
Title: Re: minisphere 2.1.6
Post by: Fat Cerberus on January 27, 2016, 03:53:25 pm
-game (single hyphen) should still be supported for legacy compatibility, however new-style --game is not, because I switched to the more Unix-y convention of:
msphere [options] <path>

This was changed in version 2.0.  I don't think it will accept an .sgm file on the command line though, due to a bug I haven't gotten around to fixing.  As a workaround for now, just pass the name of the directory containing game.sgm.
Title: Re: minisphere 2.1.6
Post by: Fat Cerberus on January 28, 2016, 01:57:49 am
Here's another screenshot.  It's starting to look very mature now. :)
Title: Re: minisphere 2.1.6
Post by: Eggbertx on January 28, 2016, 01:50:29 pm
This may have already been addressed, but have there been any plans to add mp3 and/or midi support? I have it in Linux, fully updated as of yesterday.
Title: Re: minisphere 2.1.6
Post by: Fat Cerberus on January 28, 2016, 02:19:20 pm
mp3 is impossible without relicensing minisphere as GPL (no thank you), and MIDI... I don't even know where to start with that.  For Windows I can probably just call into winmm, no idea for other platforms.
Title: Re: minisphere 2.1.6
Post by: Eggbertx on January 28, 2016, 06:16:40 pm
Oh, alright. It was more a matter of convenience. I have Audacity, so converting oggs is no problem, and I have timidity installed, so I can just use a soundfont to render a midi file as an ogg.
Title: Re: minisphere 2.1.6
Post by: Radnen on January 28, 2016, 09:06:31 pm
I think .ogg is better than .mp3 because it can play at lower bitrates with nearly the same quality, thus reducing filesize (especially useful for certain songs in midi or other soundfont formats).
Title: Re: minisphere 2.1.6
Post by: Fat Cerberus on January 28, 2016, 09:14:15 pm
That's definitely true in my experience - .ogg still sounds decent even down to 48kbps in stereo (the lowest bitrate the encoder will give you in "constant quality" mode) - try that with an mp3 and it'll be so full of compression noise as to be unlistenable.
Title: Re: minisphere 2.1.6
Post by: Fat Cerberus on January 30, 2016, 03:03:43 am
SSJ works on Linux too!
Title: Re: minisphere 2.1.6
Post by: Fat Cerberus on February 01, 2016, 02:18:44 am
I just switched minisphere's UNIX build system from SCons to make.  Not autotools, I don't have time for that. :P  It's a hand-made makefile.  The makefile allows minisphere and GDK tools to be installed in /usr/bin as well:
Code: [Select]
make && sudo make install


Also, "make dist" will make a tarball.

From there it's a simple matter to run games from the command line:
Code: [Select]
sphere game.spk
sphere game.sgm


Or to debug them (note SSJ doesn't yet show source):
Code: [Select]
ssj game.sgm
ssj --pause game.sgm
(the debugger works with SPKs too!)
Title: Re: minisphere 2.1.6
Post by: Flying Jester on February 01, 2016, 11:34:01 am

I just switched minisphere's UNIX build system from SCons to make.  Not autotools, I don't have time for that. :P


The days when Autotools made sense are mostly gone. Supporting Unix no longer means supporting Linux, Irix, Solaris, HP-UA, AIX, etc., and ignoring the hoopla, most Linux distros are very similar. The only thing you really need to do is support prefixes.

The other main use of autotools was to smooth over different libc's, but again, those days are gone. Every system either uses glibc, bsd libc (which is very similar in API to glibc), or something that tries its best to look like glibc (like musl).
Title: Re: minisphere 2.1.6
Post by: Eggbertx on February 01, 2016, 04:31:00 pm
So since Oracle either stopped or will stop working on the applet system (though thanks to Oracle's crap security, it was pretty much dead in the water a while ago) JavaSphere is a no go there, and since doing JavaSphere from scratch would be effectively reinventing the wheel, do you think it would be at all possible to port miniSphere to Android? Looking around, it doesn't look like it should be too difficult to compile Allegro stuff for Android. I'm not familiar with Allegro (I've mostly just used SDL in the past), but I'll see what I can do. Keyboard stuff might be a bit tricky, if impossible, but mouse stuff could possibly be converted to swiping and touching. So while there might be a lot of Sphere games that either won't work at all, or will have to be heavily modified for Android, I still think it could be pretty cool.
Title: Re: minisphere 2.1.6
Post by: Fat Cerberus on February 01, 2016, 04:40:44 pm
Allegro has a lot of stuff built in that I would likely have to implement myself or go looking for third-party add-on libraries to do in SDL, and it's ridiculously simple to compile static (the official Windows releases are statically linked to Allegro).  Also, SDL as far as I know is LGPL, making static linking more of an iffy proposition due to minisphere's BSD license.  Allegro is zlib licensed.  Oh, and I like the Allegro API, very C-like. :)

Allegro can be compiled on Android, and actually I wanted to do an Android build of the engine at some point in the future, but getting the build environment set up seemed like a daunting task, so I keep putting it on the back burner.  Feel free to try it yourself and let me know the results!
Title: Re: minisphere 2.1.6
Post by: Eggbertx on February 01, 2016, 07:25:56 pm
If that's the case, it might be easier to just use SDL or something. I'll look into working with one of the other cross-platform engines, or just making my own and using some code from them as reference. But I'm already working on a non-Sphere related project, so that may or may not happen. (shameless (http://github.com/eggbertx/gochan) plug (http://gochan.org))
Heck, since it's developed by Google anyway, can is compatible with Linux, OS X, Windows, Android, and iOS and it can link other non-Go libraries like SDL and GTK, I wonder how feasable a Sphere engine written in Go (http://golang.org) would be...
Title: Re: minisphere 2.1.6
Post by: Fat Cerberus on February 01, 2016, 07:30:58 pm
Do keep in mind that when I said it seemed daunting, I was developing exclusively on Windows at the time--a platform which is not really well-suited to cross-platform development. :P  Now that i have access to Linux, I might try it again at some point in the future.
Title: Re: minisphere 2.1.6
Post by: Eggbertx on February 01, 2016, 10:19:00 pm
That's fine. I haven't used it a whole lot since building it a few days ago in Linux, but it seems to run pretty well. This was more of a random idea, since Radnen suggested I look into porting JavaSphere to Android.
Title: Re: minisphere 2.1.6
Post by: Fat Cerberus on February 03, 2016, 02:47:40 pm
minisphere has a man page now!
Title: Re: minisphere 2.1.6
Post by: Flying Jester on February 04, 2016, 03:51:41 pm

Allegro has a lot of stuff built in that I would likely have to implement myself or go looking for third-party add-on libraries to do in SDL, and it's ridiculously simple to compile static (the official Windows releases are statically linked to Allegro).  Also, SDL as far as I know is LGPL, making static linking more of an iffy proposition due to minisphere's BSD license.  Allegro is zlib licensed.  Oh, and I like the Allegro API, very C-like. :)


SDL2 is zlib licensed.

SDL2 is also extremely similar to SDL1 in an API sense (to the point that a few very common functions now have dummy arguments to match the API of their SDL1 counterparts). I personally prefer SDL2's API, and if you were going to call it from Java I think it would be a much better choice since its API is very well suited to being used from an object-oriented environment. SDL2 also has full support for Android, and has a variety of features to either query the actual touch system or to just emulate a keyboard and mouse with it.

If cross-platfom is a goal, SDL1 and even SDL2 utterly annihilate Allegro, though :P
Title: Re: minisphere 2.1.6
Post by: Fat Cerberus on February 04, 2016, 04:07:09 pm
Not sure about Allegro 4, but v5 supports Android back to API level 12, iOS, and of course the big three (Windows, OSX, Linux).  Possibly BSD as well, not sure about that though.  Allegro 4 is very much a legacy platform (being designed originally for MS-DOS games and ported elsewhere over the years), v5 was rewritten from the ground up but it gets a bad rap because people assume it's just an incremental evolution of the classic Allegro.

SDL probably does support more platforms, but for something like Sphere, the 5 listed above should cover the majority of users.  Honestly though most of why minisphere is still using Allegro is inertia - it was the first thing I tried after SFML, and had everything I needed--and still does--and now that I have the whole Sphere API built on top of it (note: the Allegro 5 API is VERY similar in structure to Sphere 1.x) it would be a massive undertaking to port it to a new backend.

Speaking of porting, I backported minisphere to be able to compile against Allegro 5.0, which is what's pulled by default via apt-get and homebrew since 5.1 is "unstable".  This should make it easier for people to compile the engine now. :)
Title: Re: minisphere 2.1.6
Post by: Eggbertx on February 04, 2016, 04:33:21 pm

minisphere has a man page now!

Nice! So is this something that will be installed by make install?


Allegro 4 is very much a legacy platform (being designed originally for MS-DOS games and ported elsewhere over the years),

Wow, I knew Allegro was around during the DOS days, but I didn't know that Allegro 4 was that old.
Title: Re: minisphere 2.1.6
Post by: Fat Cerberus on February 04, 2016, 04:37:22 pm


minisphere has a man page now!

Nice! So is this something that will be installed by make install?


Yes.  There are also man pages for Cell and SSJ, but they are currently incomplete.
Title: Re: minisphere 2.1.6
Post by: Flying Jester on February 04, 2016, 06:14:28 pm
Allegro 5 sort of supports FreeBSD (no sound), and doesn't support OpenBSD at all. They say they do, but the OpenBSD people say they don't (and Allegro doesn't compile in OpenBSD 5.7+, I know from experience). Either way, they don't support the OpenBSD audio system for sure.

The other targets that are important to me are Solaris (particularly on UltraSparc) and Haiku. SDL2 supports both, although their raw Surface video backend is pretty bugged on Haiku (the Render backend and OpenGL support has no issues, though). It also has audio on BSDs. But I don't use SDL2 for audio anyway, I usually use OpenAL or OpenBSD's native sndio when on OpenBSD or Linux (which is easily the best audio backend I've found so far).

You're right, though, these are pretty obscure targets. Allegro could probably be made to support FreeBSD with PulseAudio enabled, but the BSD people hate the PulseAudio people and the PulseAudio people hate the BSD people, so I don't suspect that's a good long term plan.
Title: Re: minisphere 2.1.6
Post by: Fat Cerberus on February 06, 2016, 03:34:03 am
So I finally got Travis to actually use clang to build minisphere (turns out SCons doesn't honor the pre-existing environment, it was simple now that I have a makefile), and said compiler now seems to hate me, mostly because I like to do this:

Code: (c) [Select]

if (thing = create_foobar(...)) {
    ...
}


It's just so succinct and elegant that I got into the habit of doing it everywhere.  I can easily shut up the warnings of course, by adding another set of parentheses, but that triggers my OCD and makes the statement look uglier. :-\

I should try adding -Wall -pedantic to that command line and see if I can get clang even angrier at me. :P

Anyway, I think I'm getting close to releasing the first 3.0 beta, but I have to get SSJ to be able to locate source code properly first, and also implement breakpoints.  It's kind of useless without those. ;)
Title: Re: minisphere 2.1.6
Post by: Fat Cerberus on February 07, 2016, 01:50:18 am
I think minisphere on Windows XP is a lost cause.  I set up Windows XP x64 edition in a VirtualBox VM yesterday and tried to get minisphere going on it, but failed miserably.  I think it might just be that the MSVC 2015 C runtime doesn't like XP - everything was fine until something tried to access the file system.  Cell will report that there's no cellscript in the current directory, minisphere errors out if a game is passed on the command line, the "select a game" dialog doesn't show up at all, etc.

So yeah, I'm just going to cross XP off the list of supported platforms.  Anyone know if minisphere runs on Vista or should I just say the minimum requirement is Win7 and call it a day?

In any case, I decided not to drop the 32-bit builds after all.  There are still quite a few 32-bit Windows installs in the wild, particularly netbooks tend to be preinstalled with a 32-bit OS to counterbalance the low RAM.  Since both the 32- and 64-bit binaries are built from the same source, it's not a big deal to maintain two sets of binaries.  Speaking of which, starting with minisphere 3.0 the entire toolchain will be built for 64-bit.  In minisphere 2.x, Cell was 32-bit even in a 64-bit minisphere installation.
Title: Re: minisphere 2.1.6
Post by: Flying Jester on February 07, 2016, 02:27:45 pm
XP is more than 15 years old. If someone is disappointed with Minisphere not supporting it, I would be more disappointed they are still using XP.
Literally even Wine is better supported, more compatible, more stable, and more secure at this point.


So I finally got Travis to actually use clang to build minisphere (turns out SCons doesn't honor the pre-existing environment, it was simple now that I have a makefile)

It's easy once you know how, but I agree that SCons should really accept hints from $CC, $CXX, etc. When I started fiddling with Emscripten I had to learn how to accept environment settings, but it really should be the default to do so.


Code: (c) [Select]

if (thing = create_foobar(...)) {
    ...
}


It's just so succinct and elegant that I got into the habit of doing it everywhere.  I can easily shut up the warnings of course, by adding another set of parentheses, but that triggers my OCD and makes the statement look uglier. :-\


I sort of understand why they do this. It's because using both equality and assignment has screwy precedence order...
Code: [Select]

int x = 1;
if(x=0 == 1)
    causeNuclearWar();
else
    puppiesForEveryone();
// I hope that turned out OK!


It also is intended to show you that in such cases variable decaration will not be hoisted to the next block. But I wish they would just leave these warnings on conditionals with both assignment and equality.
Title: Re: minisphere 2.1.6
Post by: Fat Cerberus on February 09, 2016, 12:04:43 pm
Hey, could someone with a 64-bit installation of Debian or Ubuntu test out this .deb package for me?
https://drive.google.com/open?id=0BxPKLRqQOUSNVmR3SFU0QXB6ZUE

I'll be providing .deb's and tarballs for minisphere 3.0, so I figured I'd make sure everything is in order now. :)
Title: Re: minisphere 2.1.6
Post by: DaVince on February 09, 2016, 08:30:45 pm
I suppose the Allegro 5.1 PPA is mandatory for this one, since it gave me "Dependency not satisfiable: liballegro-acodec5.1" in Ubuntu's package manager.

Installed fine after adding that. The minisphere desktop entry also appeared in my Xfce menu. It opened the standard "open file" dialog; can't help but feel that showing the launcher game with sample games would be better though. (Maybe with a simple game that explains/demonstrates what this whole thing is? That could be a future step, I suppose... :P )
Title: Re: minisphere 2.1.6
Post by: Fat Cerberus on February 09, 2016, 08:42:10 pm
It's actually supposed to show the startup game, guess that's a packaging bug.  Thanks for testing!

As for Allegro 5.1, minisphere will actually compile against Allegro 5.0 now, at the cost of disabling some features--the ones I know are shaders, error copy-to-clipboard, and vertex buffers (improves Galileo performance).  But apparently debuild just sees that Allegro 5.1 is installed and declares that as the dependency.  I'll see if I can fix that before release.

By the way if you drop to command line, can you successfully execute cell and ssj?  Also check if you have man pages for minisphere, spherun, cell and ssj.
Title: Re: minisphere 2.1.6
Post by: DaVince on February 09, 2016, 08:46:35 pm
All of the commands and manpages seem to be present.

I also noticed that my dev copy of Sir Boingers likes to lock up and crash minisphere when navigating over the High Scores option. And also when in the pause menu for half a second or so. Not sure what's up with that; haven't tested if it only happens with the precompiled version or also when I compile it myself. In any case, I can't even navigate down enough to enable the sound and music...

Edit: this error when running from the command line:
Code: [Select]
fish: "minisphere ." terminated by signal SIGSEGV (Address boundary error)


Also, when trying to run the game with ssj, I get:
Code: [Select]
$ ssj .
SSJ 3.0a0 Sphere game debugger x64
A powerful JavaScript debugger for minisphere
(c) 2016 Fat Cerberus

Starting '/home/vincent/Dropbox/projects/sphere/boingers2.2/games/SirBoingers/'... OK.
Connecting to 127.0.0.1:1208... OK.
Handshaking... OK.
  Attached to minisphere 3.0a0
  Duktape v1.4.0
uncaught `TypeError: cannot read property 'origin' of undefined` at input:1
Unrecoverable error, target detached

^C⏎
Title: Re: minisphere 2.1.6
Post by: Fat Cerberus on February 09, 2016, 08:49:41 pm
Alright, I'll look into that.  It's possible there's a regression in the latest builds.  Could you PM me a copy of the game?
Title: Re: minisphere 2.1.6
Post by: DaVince on February 09, 2016, 08:51:13 pm
Sure. I also edited my above post with the error messages.
Title: Re: minisphere 2.1.6
Post by: Fat Cerberus on February 09, 2016, 08:54:15 pm
That second one is a bug in SSJ, it's trying to load the source map which isn't present unless you compile with Cell using the -d switch.  It's supposed to gracefully degrade, but apparently something fouled up there. ;)
Title: Re: minisphere 2.1.6
Post by: Fat Cerberus on February 11, 2016, 03:04:02 am
So I came up with a way to make Cell more useful.

In minisphere 2.1, compiling with cell -d will generate a debug map which maps the name of a script in the compiled package to its location in the source tree.  This aids the debugger (Sphere Studio for 2.1, SSJ for 3.0) in locating the original scripts, even if the Cellscript relocates them.

For Cell 3.0 I'm going to add a new option, --ssj2, which augments the debug info with the full original source code for all scripts.  Not only does this allow easy debugging of SPK packages even if the original source tree changes (or is moved), but it will also let you release betas compiled with --ssj2 that others can run through the debugger when errors occur to get meaningful info. :D
Title: Re: minisphere 2.1.6
Post by: Fat Cerberus on February 11, 2016, 02:36:02 pm
SSJ is starting to shape up.  I just implemented listing source code:
Code: [Select]

fatcerberus@pigcult-vm:~/src/spectacles-i$ ssj -p specs.spk
SSJ 2.99.812.1 Sphere debugger (x64)
A powerful JavaScript debugger for minisphere
(c) 2016 Fat Cerberus

spherun /home/fatcerberus/src/spectacles-i/specs.spk... OK.
connecting to 127.0.0.1:1208... OK.
verifying... OK.
: attached minisphere 2.99.812.1
: duktape v1.4.0
locating sources... OK.
: source tree /home/fatcerberus/src/spectacles-i/

>>   0: eval() at scripts/main.js:6
6: const DBG_DISABLE_TEXTBOXES = false;

eval() ssj$ l
      1 /***
      2  * Specs Engine v6: Spectacles Saga Game Engine
      3   *           Copyright (c) 2015 Power-Command
      4 ***/
      5
>>    6 const DBG_DISABLE_TEXTBOXES = false;
      7 const DBG_DISABLE_TRANSITIONS = false;
      8
      9 RequireSystemScript('mini/miniRT.js');
     10 RequireSystemScript('analogue.js');

eval() ssj$ so
>>   0: game() at scripts/main.js:30
30: mini.initialize();

game() ssj$ l
     25
     26 // game() function
     27 // This is called by Sphere when the game is launched.
     28 function game()
     29 {
>>   30 mini.initialize();
     31 analogue.init();
     32
     33 mini.Console.register('specs', global, {
     34 'exit': function() { Exit(); }

game() ssj$ q
SSJ session has been detached.
Title: Re: minisphere 3.0b1 (stable: 2.1.6)
Post by: Fat Cerberus on February 12, 2016, 02:46:54 am
After almost 2 months of development, the first beta of minisphere 3.0 is available.  Make sure to test drive SSJ and let me know of any issues.  It still needs polishing, but should be functional enough to use for actual debugging. :P

I recommend uninstalling previous versions of minisphere before trying the beta as the engine executables were renamed, so the old files may cause conflicts/confusion.  Note also that Sphere Studio is no longer included in the download--that will have to be installed separately now for anyone that wants it.  The GDK plugin is included, though, and will be picked up by Sphere Studio 1.2.

If you choose to add minisphere to the PATH, the command to run a game on the command-line is:
spherun <gamepath>

If you omit the path, it will list available command-line options.

edit: By the way, minisphere is available on Ubuntu 14.04 now, through PPA:
Code: [Select]

sudo add-apt-repository ppa:fatcerberus/minisphere
sudo apt-get update
sudo apt-get install minisphere


This probably won't work on later versions of Ubuntu, e.g. 15.10 due to Ubuntu's braindead PPA packaging system (you can't upload multiple packages with the same version number, even if they are built for a different Ubuntu version).  So I'll be posting .deb packages sometime in the near future.

edit 2: Alright, the .deb packages are now available for direct download for those who can't get it through APT.
Title: Re: minisphere 3.0b1 (stable: 2.1.6)
Post by: DaVince on February 12, 2016, 12:30:41 pm
Quote
This probably won't work on later versions of Ubuntu, e.g. 15.10 due to Ubuntu's braindead PPA packaging system (you can't upload multiple packages with the same version number, even if they are built for a different Ubuntu version)

I suppose this is why they keep appending -1, -2, etc. to the end of the version number... You might want to do that for the 15.10 repo, since that's probably the other Ubuntu version a lot of people (including me :P ) will be running.

Will just get the deb file now.
Title: Re: minisphere 3.0b1 (stable: 2.1.6)
Post by: Fat Cerberus on February 13, 2016, 12:25:35 am
Heads up, DaVince: I just realized why I originally made the switch to Allegro 5.1.  There are some pretty nasty bugs in 5.0, as I was just reminded by my cousin beta testing it.  Notably, sometimes playing sounds will completely lock up the engine, from which the only way out is to kill the process.  It's a hard deadlock due to a race condition in Allegro's streaming audio code, nothing minisphere can do to work around it. :(

If worse comes to worst I'll have to statically link Allegro for the final Linux release.  This will likely disqualify it from official Debian/Ubuntu packaging, but it'd just be a stopgap measure until a new stable Allegro release comes out.

I hope they release Allegro 5.2 soon... I'm not really up for trying to port the whole thing to SDL right now. :P  Although it might be an interesting experiment to do in my spare time, I'd like to try to get 3.0 out the door first. :P
Title: Re: minisphere 3.0b1 (stable: 2.1.6)
Post by: DaVince on February 13, 2016, 01:08:38 pm
Yeah, I remember those bugs too. Thing is, the engine hasn't crashed where it would before, so I figured things have gotten at least a *little* better with Allegro 5.0 somehow. Still, I can imagine the earliest minisphere would be able to be included in any official repo is Ubuntu 16.04 or so. On Ubuntu and related distros, anyway.
Title: Re: minisphere 3.0b1 (stable: 2.1.6)
Post by: Fat Cerberus on February 15, 2016, 11:22:47 pm
New SSJ feature coming in minisphere 3.0b2: the Examine command.  It's like Eval, but more verbose and shows Duktape runtime data, which can help in identifying objects:

Normal Eval:
Code: [Select]

scripts/main.js ssj$ e Image.prototype
= {
   prop "toString" = {...}
   prop "height" = { get }
   prop "width" = { get }
   prop "blit" = {...}
   prop "blitMask" = {...}
   prop "createSurface" = {...}
   prop "rotateBlit" = {...}
   prop "rotateBlitMask" = {...}
   prop "transformBlit" = {...}
   prop "transformBlitMask" = {...}
   prop "zoomBlit" = {...}
   prop "zoomBlitMask" = {...}
}


Examine:
Code: [Select]

scripts/main.js ssj$ x Image.prototype
= {
   meta "heapptr" = { heap:"0000019a70a80620h" }
   meta "heaphdr_flags" = 1342177408
   meta "heaphdr_type" = 2
   meta "refcount" = 2
   meta "reachable" = 0
   meta "temproot" = 0
   meta "finalizable" = 0
   meta "finalized" = 0
   meta "readonly" = 0
   meta "extensible" = 1
   meta "constructable" = 0
   meta "bound" = 0
   meta "compiledfunction" = 0
   meta "nativefunction" = 0
   meta "bufferobject" = 0
   meta "thread" = 0
   meta "array_part" = 0
   meta "strict" = 0
   meta "notail" = 0
   meta "newenv" = 0
   meta "namebinding" = 0
   meta "createargs" = 0
   meta "envrecclosed" = 0
   meta "exotic_array" = 0
   meta "exotic_stringobj" = 0
   meta "exotic_arguments" = 0
   meta "exotic_dukfunc" = 0
   meta "exotic_proxyobj" = 0
   meta "class_number" = 10
   meta "class_name" = "Object"
   meta "internal_prototype" = { obj:"0000019a6e86eba0h" }
   meta "props" = { ptr:"0000019a6e911750h" }
   meta "e_size" = 14
   meta "e_next" = 14
   meta "a_size" = 0
   meta "h_size" = 0
   prop "toString" = { obj:"0000019a70a67e30h" }
   prop "height" = { get: { obj:"0000019a70a68070h" }, set: null }
   prop "width" = { get: { obj:"0000019a70a67a70h" }, set: null }
   prop "blit" = { obj:"0000019a70a68430h" }
   prop "blitMask" = { obj:"0000019a70a68010h" }
   prop "createSurface" = { obj:"0000019a70a68af0h" }
   prop "rotateBlit" = { obj:"0000019a70a681f0h" }
   prop "rotateBlitMask" = { obj:"0000019a70a67ad0h" }
   prop "transformBlit" = { obj:"0000019a70a68610h" }
   prop "transformBlitMask" = { obj:"0000019a70a67e90h" }
   prop "zoomBlit" = { obj:"0000019a70a68850h" }
   prop "zoomBlitMask" = { obj:"0000019a70a67c50h" }
}
Title: Re: minisphere 3.0b2 (stable: 2.1.6)
Post by: Fat Cerberus on February 22, 2016, 02:03:27 pm
minisphere 3.0b2 for Windows is up, with a vastly improved SSJ debugger. :D  I recommend testing it out on real Sphere games - by my guess, minisphere 3.0 Final is getting pretty close so testing would be much appreciated.

I'll post the tarball and deb downloads later.
Title: Re: minisphere 3.0b2 (stable: 2.1.6)
Post by: Fat Cerberus on February 22, 2016, 02:52:54 pm
@DaVince The 64-bit deb package is up.  Could you do some quick testing?  SSJ has a full manpage now if you need a primer. ;)
Title: Re: minisphere 3.0b2 (stable: 2.1.6)
Post by: DaVince on February 23, 2016, 12:26:18 am
Both seems to be working great, for as far as I've tested them! (Only tested SSJ as far as Print/DebugPrint and backtracing a game that crashes on it (an old special Kefka's Revenge winter demo I have yet to upload anywhere).)

The crash happened in "undefined:0", but at least the backtrace ensured it was clear it was in the map engine and exactly what line in the scripts started that map engine with which map. That is really useful. Thanks a ton for making this! :)

Edit: should say, though, if this could somehow report which line/script in the map file is doing that, that would be cool, because... the map it errored out on has like fifty entities.
Title: Re: minisphere 3.0b2 (stable: 2.1.6)
Post by: Fat Cerberus on February 23, 2016, 12:43:03 am
If the error is in an embedded map script, it should be reporting the filename as something like Test.rmp:onEnter or Test.rmp:Entity1:Talk.  You mean it's not doing that?

Note that it doesn't (currently) show source code for embedded map scripts, although that may be possible to do.
Title: Re: minisphere 3.0b2 (stable: 2.1.6)
Post by: Fat Cerberus on February 23, 2016, 01:06:25 am

The crash happened in "undefined:0", but at least the backtrace ensured it was clear it was in the map engine and exactly what line in the scripts started that map engine with which map. That is really useful. Thanks a ton for making this! :)

Edit: should say, though, if this could somehow report which line/script in the map file is doing that, that would be cool, because... the map it errored out on has like fifty entities.


I just realized, if it says "undefined:0" that means the error was thrown by a Sphere API (e.g. Abort()), not directly by the script.  That's something I need to fix before the final release: It should automatically select the next stack frame up if the error occurs in an API function (the API is throwing it on behalf of the game anyway).  For now if it drops you into "undefined:0" just enter the command "up" to switch to the function that actually caused the error.

And hey, no problem - A symbolic debugger is something I always wished Sphere had and the lack of one always held it back as serious game development tool for me.  Finally having it at my disposal is awesome--even more awesome because I wrote it. ;D
Title: Re: minisphere 3.0b2 (stable: 2.1.6)
Post by: DaVince on February 23, 2016, 01:29:43 am

I just realized, if it says "undefined:0" that means the error was thrown by a Sphere API (e.g. Abort()), not directly by the script.  That's something I need to fix before the final release: It should automatically select the next stack frame up if the error occurs in an API function (the API is throwing it on behalf of the game anyway).  For now if it drops you into "undefined:0" just enter the command "up" to switch to the function that actually caused the error.

Oh! You're right! I didn't see it at all, but it *does* say this:

Code: [Select]
FATAL UNCAUGHT - TypeError: number required, found '0' (stack index 1)
   at: [nutcracker1 : queued script]:1
=> # 0: SetPersonFrame() at undefined:0
'undefined': source file could not be located.


I guess what threw me off is that it's not giving the map file name, so I wasn't looking properly for the "nutcracker : queued script" bit.
Title: Re: minisphere 3.0b2 (stable: 2.1.6)
Post by: Fat Cerberus on February 23, 2016, 01:37:04 am
Dammit, I think I'm going to have to bite the bullet in minisphere 3.0 and allow API functions to accept 0/1 in place of true/false.  Quite a few Sphere games seem to do that but minisphere doesn't like it because it's not actually a JS boolean value.  I suspect that's the cause of your error.

The reason it doesn't show the map file name is because at the time the script is queued (using QueuePersonScript) only the person name is known.  I could probably fix that by tracking which map transient persons are created for.
Title: Re: minisphere 3.0b2 (stable: 2.1.6)
Post by: Fat Cerberus on February 23, 2016, 09:45:00 am
Hm, on second look it actually looks like error is that the game passed a string for the second argument to SetPersonFrame() which expects a number in that slot.  Since this is based on Kekfa's Revenge code I shouldn't really be surprised. :P
Title: Re: minisphere 3.0b2 (stable: 2.1.6)
Post by: Fat Cerberus on February 23, 2016, 02:19:54 pm
Next (pre)release of minisphere I want to add the ability to send source to the debugger directly from the running game so that:



My goal is to have minisphere 3.0 go gold on the first anniversary of the minisphere 1.0 release, March 28.  So that gives me about a month.  Looks like 2016 will be the year of the Sphere debugger! :)
Title: Re: minisphere 3.0b2 (stable: 2.1.6)
Post by: Fat Cerberus on February 25, 2016, 01:14:44 am
@DaVince
I implemented source download in SSJ (after solving a nasty segfault mentioned in the Programming board that ruined my night...), and using it, proceeded to track down the cause of your KR crashes:

Code: [Select]

C:\Users\fatce\Documents\Sphere Studio\Projects>ssj kr
SSJ 3.0a3 minisphere Console Debugger (x64)
a powerful JavaScript debugger for minisphere
(c) 2016 Fat Cerberus

starting C:/Users/fatce/Documents/Sphere Studio/Projects/kr/... OK.
connecting to 127.0.0.1:1208... OK.
verifying... OK.
querying target... OK.
    game title: Kefka's Revenge
    author: KRTeam

=> # 0: eval() at scripts/game.js:12
12: GetSystemFont().drawText(15,10,"LOADING...");

scripts/game.js:12 eval()
(ssj) c
FATAL UNCAUGHT - TypeError: number required, found '83' (stack index 1)
    at: maps/Aresidence1.rmp/onEnter:2
=> # 1: global() at maps/Aresidence1.rmp/onEnter:2
2: SetPersonX('daughterslaughter', '83');

maps/Aresidence1.rmp/onEnter:2 global()
(ssj) l
      1: dispTown = "Family home";
=>    2: SetPersonX('daughterslaughter', '83');
      3: SetPersonX('workdude', '277');
      4: SetPersonY('workdude', '212');
      5: SetPersonDirection('workdude', 'west');
      6: IgnorePersonObstructions('kid', true);


Unfortunately KR does things like this a lot, which is what makes it so hard to support.  But now you can see source for map scripts directly in the debugger, which is awesome!
Title: Re: minisphere 3.0b2 (stable: 2.1.6)
Post by: DaVince on February 25, 2016, 01:35:38 pm
Wow, thanks! I really don't think you needed to 'fix' this, since it would kinda promote bad coding practice, but... hey, you did it! :P
Title: Re: minisphere 3.0b2 (stable: 2.1.6)
Post by: Fat Cerberus on February 25, 2016, 01:39:04 pm
I didn't fix the KR bug if that's what you mean - I mostly just wanted to track down the cause since I wasn't able to do this even with the Sphere Studio debugger.  Since the classic map engine is still supported in minisphere, it was kind of dumb that you couldn't see map code in the debugger.  Now you can!
Title: Re: minisphere 3.0b2 (stable: 2.1.6)
Post by: Fat Cerberus on February 25, 2016, 02:13:03 pm
It even works for scripts set up at runtime (e.g. update scripts), for example here it shows the GenCommands stub installed by analogue.js:

Code: [Select]

src/main.js:53 game()
(ssj) w
   # 0: global() at maps/Testville.rmp/maggie/genCommands:1
1: analogue.runPersonEvent(GetCurrentMap(), 'maggie', 'generator');
Title: Re: minisphere 3.0b2 (stable: 2.1.6)
Post by: DaVince on February 25, 2016, 03:33:26 pm

I didn't fix the KR bug if that's what you mean - I mostly just wanted to track down the cause since I wasn't able to do this even with the Sphere Studio debugger.  Since the classic map engine is still supported in minisphere, it was kind of dumb that you couldn't see map code in the debugger.  Now you can!

Ah, okay! That makes a LOT more sense. In any case, this just makes it more powerful!
Title: Re: minisphere 3.0b2 (stable: 2.1.6)
Post by: Fat Cerberus on February 25, 2016, 05:11:49 pm
Another useful feature I added is that if execution stops in a system call, SSJ will automatically select the next stack frame up that actually contains JS code.  This avoids the need to do "up" first.
Title: Re: minisphere 3.0b3 (stable: 2.1.6)
Post by: Fat Cerberus on February 29, 2016, 02:50:10 am
Just posted another beta, minisphere 3.0b3.  SSJ has a brand-new feature: It pulls source code for the game being debugged directly from minisphere.  This means you can debug games, even those packaged as SPKs, without having access to the original source tree: So long as the scripts weren't minified or otherwise mangled before being compressed, you can now debug any Sphere game past, present and future.

I also enhanced the output of SSJ's "examine" command to make it more useful this release: It now shows property attributes for all properties and produces a nice listing like Unix ls -l:

Code: [Select]

scripts/main.js:30 game()
ssj: x GetGameManifest()
ewc- name                "Spectacles: Bruce's Story"
ewc- author              "Fat Cerberus"
ewc- summary             "Follow Scott Starcross in his quest to stop the Primus."
ewc- resolution          "320x240"
ewc- script              "scripts/main.js"
ewc- minimumPlatform     {obj:"00000187b9161c10h"}
ewc- frameRate           60
ewc- sceneRenderPriority 99
ewc- logPath             "~usr/Spectacles Saga/Console Log.txt"
ewc- consoleBuffer       1000
ewc- consoleLines        10
ewc- disableAnimation    false
ewc- disableBattles      false
ewc- disableSplash       true
ewc- disableTitleScreen  true
ewc- directory           "C:/src/spectacles-i/dist/"


That's "ewc" for "Enumerable, Writable, Configurable".  The last slot which is clear here is "a" for Accessor (a property with a getter/setter).  All properties are listed, even non-enumerable ones.  This is in contrast to eval which only lists enumerable properties.  If you do examine global be prepared for an absolutely massive listing!
Title: Re: minisphere 3.0b3 (stable: 2.1.6)
Post by: Fat Cerberus on March 03, 2016, 01:14:51 am
@DaVince: Have you tried the new beta yet?  SSJ is much more useful now. :) (there might be a few crashes which I've since fixed in the git repo)
Title: Re: minisphere 3.0b3 (stable: 2.1.6)
Post by: DaVince on March 03, 2016, 02:52:53 pm
Not yet, I haven't really had much time to dedicate to game dev lately! Also not sure where/how I would use all this shiny new stuff, will definitely have to read the manpages and try things out.

To be honest I'm not used to debuggers in general, so it's going to take a bit for me to get used to it all. :)
Title: Re: minisphere 3.0b3 (stable: 2.1.6)
Post by: Fat Cerberus on March 03, 2016, 02:59:57 pm
Cell is most likely going to get an overhaul before the official release (March 28), the codebase is not very modular making any changes difficult, and writing a Cellscript isn't particularly intuitive even with the manpage.

SSJ probably won't change any further, I'm very satisfied with how it turned out. :D

minisphere itself hasn't gotten many changes at all, the 2.1 codebase was already quite mature.  The only major change I made was to support source code download by the debugger.  Other than that it's just minor bug fixes.
Title: Re: minisphere 3.0b3 (stable: 2.1.6)
Post by: Fat Cerberus on March 04, 2016, 03:08:07 am
Hm, on second thought I think I'll hold off on that Cell overhaul until v3.1.  Development time would be better spent improving the Sphere Studio debugger UI, which has kind of fallen behind compared to SSJ's feature set, particularly w.r.t. variable and object inspection.
Title: Re: minisphere 3.0b4 (stable: 2.1.6)
Post by: Fat Cerberus on March 04, 2016, 04:56:55 pm
I just posted the fourth and final beta for minisphere 3.0.  This one fixes a few SSJ crashes and has vastly improved object inspection in Sphere Studio.  See screenshot attached.

The rest of the time before release will be spent improving and polishing off the documentation, particularly the Linux manpages which are still a bit incomplete.
Title: Re: minisphere 3.0rc1
Post by: Fat Cerberus on March 06, 2016, 02:14:34 am
As the minisphere 3.0 release is only 3 weeks away, I decided to do a code freeze.  minisphere 3.0rc1 has been posted, and if no showstopping bugs are found, that build will be officially released as minisphere 3.0.0 on March 28.  I took down the links for 2.1.6 to encourage testing the new version.  As with all major minisphere releases, the previous one will remain available in the Spherical download repository.

Sphere Studio is no longer bundled: For Windows users, it is highly recommended to download Sphere Studio 1.2.1 as a companion to minisphere 3.0.

To the Spherical regulars:
I know you guys don't have much free time, but if some testing could be squeezed in on the RC before the 28th it would be much appreciated.  I will keep an eye out for bugs myself in the meantime.
Title: Re: minisphere 3.0rc1
Post by: N E O on March 06, 2016, 10:40:10 pm
I'll try to find time over the weekend to give it a go on Mac.
Title: Re: minisphere 3.0rc1
Post by: Fat Cerberus on March 06, 2016, 10:49:11 pm
Thanks, NEO.  By the way you don't need to build Allegro manually after all: It turns out Allegro 5.1 is available through homebrew, you just have to use the --devel switch:
https://github.com/fatcerberus/minisphere/blob/master/INSTALL.md#mac-os-x
Title: Re: minisphere 3.0rc2
Post by: Fat Cerberus on March 07, 2016, 01:12:56 am
Well, now we're at release candidate #2.  There was a change I had to make to SSJ otherwise it would have driven me bonkers, the help command didn't recognize abbreviated command names.  Now it does!

...yeah, I'm bad at this whole "code freeze" thing. :P
Title: Re: minisphere 3.0rc2
Post by: DaVince on March 07, 2016, 07:13:05 pm
Testing it out! It seems VERY stable; hasn't crashed once, even with the deb build that uses allegro 5.0. It's also fast, so far.
Now just coding random stuff (see attached file :P) and coming across the following:

I used TypeScript for a bit and it seemed to work fine. Classes worked properly and stuff. :)

LoadSound("it file").play(true) didn't work, but I think I remember this being an Allegro 5.0 specific problem, not a 5.1 one. Similarly, I think music keeps looping even if I tell it not to (there are several tunes in my test game and it's supposed to pick a random new one after finishing one song, but it always sticks to the same one tune it started out with).

surface.filledCircle(), and probably the other circle functions too, only accept integers. It errors out if I give it any radius that is a floating point value. It also doesn't seem to antialias them even when I pass true to the antialias argument.

Is there any way for the screenshot function to take a screenshot at the game's native size rather than the window size? I personally always thought that Sphere screenshotting at the native size was a strength, because it makes for really easy quick screen capping.

Finally... I still haven't played around with any of the new functions yet, other than trying TypeScript for a little bit. This is because this thread is huge and I don't know where I could find a list of the new functions specifically!
Title: Re: minisphere 3.0rc3
Post by: Fat Cerberus on March 07, 2016, 07:26:03 pm
You'll have to look through the changelog to see what's new.  That should be much more manageable than trying to sift through a nearly 70-page thread! :P  Also check minisphere-api.txt, I make it a point to add all new APIs to that file.  It's missing some Sphere 1.x stuff mostly related to the map engine, but it's comprehensive otherwise.

Thanks for the testing, especially trying out TypeScript, I didn't expect that. :)

No antialiasing on circles is due to Allegro limitations, it's a known issue that I don't have any fix for at present.  Floating point radii causing an error is unexpected though, I'll have to look into that.  As for the screenshots: This is implemented by taking a snapshot of the backbuffer.  For low resolutions, the backbuffer is actually enlarged and all rendering is run through a transformation matrix.  So getting an actual-size screenshots is actually pretty difficult.  At one point I tried to have it scale down the screenshots before saving, but that caused loss of quality, so I decided to leave it as-is.

I'll look into the music issues, I think I touched the Audialis code during 3.0 development at some point and may have fouled something up.
Title: Re: minisphere 3.0rc3
Post by: Fat Cerberus on March 07, 2016, 07:57:06 pm
@DaVince: I did some digging.  IT looping was an Allegro bug:
https://github.com/fatcerberus/minisphere/issues/17

One of the many reasons I made the switch to A5.1 for the Windows builds.  Unfortunately most of the bugs fixed in 5.1 never got backported...
Title: Re: minisphere 3.0rc3
Post by: DaVince on March 07, 2016, 09:18:36 pm
Thanks for the tips. And yeah, I was pretty sure that was a 5.0 bug. I'm obviously testing the 64-bit deb so that basically means I'm stuck with 5.0 until April. :P
Title: Re: minisphere 3.0rc2
Post by: Fat Cerberus on March 07, 2016, 09:46:34 pm
Just noticed this:


LoadSound("it file").play(true) didn't work, but I think I remember this being an Allegro 5.0 specific problem, not a 5.1 one.


This is not a bug, per se, and actually nothing to do with Allegro.  It's probably by sheer accident that it even works in Sphere 1.5.  What happens is: You're loading a sound, calling .play() on it and then immediately losing your reference to the sound object.  The garbage collector then finds it, sees it's unreachable, and finalizes it.  I'm guessing the old SpiderMonkey in Sphere 1.5 had a pretty lazy garbage collector and didn't free objects immediately, so that little trick ends up working.  Duktape primarily uses reference counting, so the second you lose the last reference, the refcount hits zero and the object is freed.

Working around it is possible: I could probably have the engine temporary bump the refcount of the sound while it's playing so that it doesn't get freed unless it stops (or the engine is closed).  That shouldn't be too difficult.  Sounds aren't intrinsically linked to the lifetime of the JS Sound object--they have their own independent refcount and everything.  That was done to allow assets to be passed around internally without causing segfaults galore when their JS objects go out of scope... those were just loads of fun to debug during early engine development. :P
Title: Re: minisphere 3.0rc3
Post by: DaVince on March 07, 2016, 10:07:31 pm
Actually? Don't fix that. In hindsight, what you said about throwing the reference away, it might actually be good for the engine to fail at accomplishing this so people will be forced to do it the 'right' way instead. :)
Title: Re: minisphere 3.0rc3
Post by: Fat Cerberus on March 08, 2016, 01:02:30 am
Huh, weird.  I changed the line below to remove the Math.round():

Code: (javascript) [Select]

this.size = Math.round(4+Math.random()*7);


And it still worked fine on my end, drew the balls and everything.  What kind of error are you seeing when giving a non-integer radius?

By the way, a tip: minisphere has a far better random number generator than Math.random() built right in, based on Mersenne Twister and allowing manual reseeding.  In this case what you'd want is:
Code: [Select]

RNG.range(min, max);

  Returns a random integer uniformly distributed within the range [min,max].


To wit:
Code: (javascript) [Select]

this.size = RNG.range(4, 11);


There's also a drop-in replacement for Math.random, called RNG.random, along with a bunch of others (search for "RNG" in the API doc).  My favorite is RNG.normal - it gives you a random number from a normal distribution with a range of about 6 standard deviations.  I use it in Specs to calculate when bosses change phases.  I was going to use it for damage calculation too, but then I figured the critical hits from that would be brutally unfair and decided against it. :P
Title: Re: minisphere 3.0rc3
Post by: Fat Cerberus on March 08, 2016, 01:22:51 pm

Actually? Don't fix that. In hindsight, what you said about throwing the reference away, it might actually be good for the engine to fail at accomplishing this so people will be forced to do it the 'right' way instead. :)


I fixed it anyway.  While it's probably bad to use that for playing looping music (since you can then never stop it), it's useful to be able to throw away references to sound effects and still have them finish--say you have a function that loads and plays a sound then returns: the sound object goes out of scope but you still want it to play through.  Without the fix you'd have to keep the reference somewhere globally accessible, which adds extra complexity (and prevents it from being GC'd ever).
Title: Re: minisphere 3.0rc3
Post by: DaVince on March 08, 2016, 02:34:09 pm
Quote
What kind of error are you seeing when giving a non-integer radius?

...Okay, I don't understand why, but suddenly it IS accepting the value. I don't know what changed that made it work, but, well, if I encounter the specific error message again I'll let you know right away.

Also, thanks for the RNG tip! That seems a lot better. I'll use it right away. Seems to work great! :)

Speaking of the API doc - where does the deb drop this file, anyway? It doesn't seem to be in /usr/share/doc/minisphere/...
Title: Re: minisphere 3.0rc3
Post by: Fat Cerberus on March 08, 2016, 02:36:50 pm
Hm... I'll have to check the makefile, it might not be installed on Linux.  In the meantime:
https://github.com/fatcerberus/minisphere/blob/master/docs/minisphere-api.txt
Title: Re: minisphere 3.0rc3
Post by: DaVince on March 08, 2016, 02:45:04 pm
Thanks! I'm seeing some things I can replace my old code with already. (Like LoadSound.)
Title: Re: minisphere 3.0rc3
Post by: Chad Zechs on March 08, 2016, 03:22:29 pm
Gentlemen. I performed a fresh install of minisphere 3.0rc3 and Sphere Studio 1.2.1, but I get an error when trying to create a new project when it tries to find a directory that doesn't seem to exist "C:\Program Files\minisphere\assets\template"

(http://i.imgur.com/w0XqinQ.png)

Is this due to something I'm doing wrong? Hope it's not wrong of me to post this on this topic.

Thanks for any assistance you can provide. I'll probably continue to fkc around with settings.

Edit: Forgot a part of my problem, bolded it haha...
Title: Re: minisphere 3.0rc3
Post by: Fat Cerberus on March 08, 2016, 03:34:38 pm
That might be a packaging bug on my part.  If so, my bad.  I'll fix it and post 3.0rc4, but in the meantime you can unzip this file into C:\Program Files\minisphere to hopefully make it work:
https://drive.google.com/open?id=0BxPKLRqQOUSNalNkdml5R1YxdXc

Never feel bad about reporting bugs - I'm always willing to fix them and this is the reason it's a release candidate and not the final release yet. :)
Title: Re: minisphere 3.0rc3
Post by: Fat Cerberus on March 08, 2016, 03:39:07 pm
@DaVince: I also recommend looking into using the S2GM format if you're going to commit to minisphere - Unlike normal game.sgm you can put arbitrary properties into it and access them using GetGameManifest().  Useful for keeping debug settings and such in a central location. 8)
Title: Re: minisphere 3.0rc3
Post by: Chad Zechs on March 08, 2016, 03:57:13 pm

That might be a packaging bug on my part.  If so, my bad.  I'll fix it and post 3.0rc4, but in the meantime you can unzip this file into C:\Program Files\minisphere to hopefully make it work:
https://drive.google.com/open?id=0BxPKLRqQOUSNalNkdml5R1YxdXc

Never feel bad about reporting bugs - I'm always willing to fix them and this is the reason it's a release candidate and not the final release yet. :)

Created project without it yelling at me, so... so far so good :) Thanks Lord English!!

Edit:
Trying to test project to see if it runs (I just left main.js to what it defaults to just to make sure it runs) and I receive the following lovely message in the Build Log:
Code: [Select]
processing Cellscript.js rule 'default'... 
  ERROR: no Cellscript rule named 'default'
================== Failed to build: Platscrol ==================


I look at Cellscript.js and nothing in there is referencing default, so I'm not sure where the disconnect is here :P
Title: Re: minisphere 3.0rc4
Post by: Fat Cerberus on March 08, 2016, 05:37:04 pm
Yeah, that's a bug in the Cellscript template.  I fixed it in 3.0rc4 (which is up).  For the already-created project you can get it to work by changing $sphere to $default at the top of the file.
Title: Re: minisphere 3.0rc3
Post by: DaVince on March 08, 2016, 06:53:02 pm

@DaVince: I also recommend looking into using the S2GM format if you're going to commit to minisphere - Unlike normal game.sgm you can put arbitrary properties into it and access them using GetGameManifest().  Useful for keeping debug settings and such in a central location. 8)

Me using SGM or S2GM is actually just directly related to which editor I use since I just let it do the work. So I'm sure this will change in the future... For now, I'm good. :P
Title: Re: minisphere 3.0rc4
Post by: Fat Cerberus on March 08, 2016, 07:17:16 pm
Ah, I guess you're still using the 1.5 editor.  I'll take another look at getting Sphere Studio to run on Linux once 3.0.0 is officially out the door.  That's really the definitive experience for minisphere development and it sucks that you have to miss out on it!

I uploaded 3.0rc4.  It installs the API docs in /usr/share/doc/minisphere now.  It turns out I never added those to the makefile because I didn't know the proper place to install them, so thanks for the tip on that. :)

I'm also hoping this is the last RC.  Thanks to my OCD I'm up to rc4 already, even Microsoft doesn't put out this many release candidates. (That doesn't excuse you from testing duty mind you! ;) )
Title: Re: minisphere 3.0rc4
Post by: Flying Jester on March 08, 2016, 07:30:10 pm
I'm also still interested in Sphere Studio on Linux/Unix!

I still tinker with Sphere-like engines from time to time, and I use a number of its formats in my other projects. I would be interested in helping debug/test it out, as well!
Title: Re: minisphere 3.0rc4
Post by: DaVince on March 09, 2016, 01:24:18 am
I've been using some new functions and object prototypes (new Font, new Color etc) and their methods and properties and it's all been working great so far.

One thing I did notice that's a thing to fix in a new minor version is that when I made a 1280x720 game it stretched in fullscreen without any regard to the aspect ratio on this 1280x1024 monitor. Result is the screenshot. (By the way, what's with the randomized filename for the screenshot? Game-name-timestamp-01.png would seem a lot more intuitive to me...)
Title: Re: minisphere 3.0rc4
Post by: Fat Cerberus on March 09, 2016, 01:30:50 am
The way minisphere's fullscreen scaling works (remember what I said about enlarged backbuffer and transformation matrices above?), it's actually a bit difficult to do letterboxing without ending up with weird artifacts where stuff gets drawn outside the screen, so I got lazy and just stretched it.  I do want to fix it eventually, but the 3.0 release is already quite big enough and I'm still busy fixing little bugs I find (SSJ's "help" command crashes in 3.0rc4, for instance), so yeah, that might have to wait till v3.1 or something.

As for the randomized screenshots, that was again due to laziness--I didn't feel like formatting a timestamp at the time so I just used RNG.string(), hehe.  I'll fix it before release.
Title: Re: minisphere 3.0rc4
Post by: DaVince on March 09, 2016, 04:48:39 pm
Okay, I can get the aspect ratio thing. That sounds pretty troublesome! (Maybe an easy solution is to force drawing two black rectangles in the appropriate areas? In any case, future feature!)

I've started actually developing a full game now, and it will specifically target and use minisphere functions whenever appropriate. I'm sure I'll be using the debugger too. I probably won't encounter many problems while this RC is happening, but at the very least there'll probably be feedback I can give you at a later stage. :)

Edit: tried creating an s2gm file according to the API documentation, but I'm getting an error when trying to run the game:
Code: [Select]
minisphere was unable to load the game manifest or it was not found.  Check to make sure the directory above exists and contains a valid Sphere game.


My s2gm looks like this, which I'm guessing is probably out of date? It's directly copied and modified from minisphere-api.txt...
Code: [Select]
{
  name: "A Game",
  author: "DaVince & SarahB",
  summary: "Our description goes here",
  resolution: '1280x1024',
  script: 'scripts/main.js',

  minimumPlatform: {
    recommend: "minisphere 3.0 or later",
    apiVersion: 3.0,
    extensions: [
      "sphere-legacy-api", //By the way, do I need these?
      "sphere-obj-constructors",
      "sphere-obj-props",
      "sphere-map-engine",
      "sphere-rng-object",
      "sphere-s2gm",
      "sphere-spherefs",
    ]
  }
}
Title: Re: minisphere 3.0rc4
Post by: Fat Cerberus on March 09, 2016, 05:02:45 pm
You only need the extensions you actually use, the engine will check them on startup to see if they're supported (minisphere-api.txt has the full list).  As for API version, that should be 2.0.  That refers to Sphere 2.0, not the specific engine.  I'll add a note in the documentation clarifying that.

The error message is wrong though (it should be something like unsupported engine), I'll look into that.
Title: Re: minisphere 3.0rc4
Post by: DaVince on March 09, 2016, 05:12:04 pm
I see. Even setting that back to 2.0 makes it return the same error message, though. I'm not sure why. Also, it can't hurt to have the full list of extensions in there, right? I think I'll be using pretty much all of them anyway. (Except for maybe legacy-api, but that depends on what that specific extension includes.)
Title: Re: minisphere 3.0rc4
Post by: Fat Cerberus on March 09, 2016, 05:17:56 pm
I'd include legacy-api just to be safe.  It's basically a catch-all for any functions directly taken from the Sphere 1.x API that weren't reassigned to a Sphere 2.0 extension, so I don't really have a concrete list of what it covers. :P  One thing for future work would be to standardize exactly what constitutes Sphere 2.0 and go from there.  Ah well, something for another day.

Anyway, I'll look into the S2GM issue tonight.  Thanks for testing as always. :D
Title: Re: minisphere 3.0rc4
Post by: Fat Cerberus on March 09, 2016, 07:19:49 pm
Okay, that's a documentation bug and is on me. :-[  There are actually several things wrong with the example: JSON requires property names to be quoted, double quotes used throughout, and doesn't allow a trailing comma after the last array entry or property.  Here's a properly-formatted file:

Code: (javascript) [Select]

{
    "name": "Hunger-Pig Feeder 5000",
    "author": "Some Weird Guy",
    "summary": "Get eaten by the hunger-pig in your closet",
    "resolution": "3840x2160",  // in awesome 4K HD!
    "script": "scripts/main.js",

    "minimumPlatform": {
        "recommend": "minisphere 3.0 or later",  // note: for display purposes only
        "apiVersion": 2.0,
        "extensions": [
            "sphere-legacy-api",
            "sphere-obj-constructors",
            "sphere-obj-props",
            "sphere-map-engine",
            "sphere-rng-object",
            "sphere-s2gm",
            "sphere-spherefs"
        ]
    }
}


I'll update the documentation.  I also added a log line when an S2GM file can't be parsed to avoid confusion in the future.
Title: Re: minisphere 3.0rc4
Post by: Fat Cerberus on March 09, 2016, 07:38:41 pm
Follow-up note: JSON also doesn't allow comments. :o
Title: Re: minisphere 3.0rc4
Post by: DaVince on March 09, 2016, 07:49:26 pm
That made it work, thanks! :) Man, I didn't know JSON parsing was so much stricter, I thought it was pretty much just a JS object...

On a related note, the deb package associates sgm files with minisphere (at least, I *think* so... if I didn't do this before by myself). But not the s2gm. That opens with a text editor by default, and is recognized as a regular text file (so even associating it with minisphere will make *all* text files associate to minisphere).
Title: Re: minisphere 3.0rc4
Post by: Fat Cerberus on March 09, 2016, 07:56:59 pm
That's weird... the debs are built using pbuilder-dist which just runs "make" and "make install" in a chroot and then packages the installed binaries.  I didn't add any file association logic to it.  Somehow the package manager must know that .sgm files should be associated with minisphere, but I have no idea how. ???

And yeah, JSON is ridiculously strict.  I could have just eval'd the manifest but that leaves open the possibility of making malicious (but ultimately harmless of course) S2GMs that enter infinite loops or other such shenanigans.  Since it's just metadata, I figured parsing it as pure JSON was better.
Title: Re: minisphere 3.0rc4
Post by: Flying Jester on March 09, 2016, 08:04:48 pm

Follow-up note: JSON also doesn't allow comments. :o


The "traditional" way around this is adding "comment" elements to objects.
Some parsers accept comments anyway, though. I personally wish they all did  :-\
Title: Re: minisphere 3.0rc4
Post by: Fat Cerberus on March 09, 2016, 08:07:51 pm
We have Doug Crockford to blame for that.  He was apparently all "OH NOES people might put nonstandard parsing directives in the comments!!!1111"
Title: Re: minisphere 3.0rc4
Post by: DaVince on March 09, 2016, 08:28:53 pm
Quote
That's weird... the debs are built using pbuilder-dist which just runs "make" and "make install" in a chroot and then packages the installed binaries.  I didn't add any file association logic to it.  Somehow the package manager must know that .sgm files should be associated with minisphere, but I have no idea how.

Never mind me, then -- I might have associated the filetype myself at some point. It works perfectly and is definitely something to add by default if you ask me. :)
Title: Re: minisphere 3.0rc4
Post by: Fat Cerberus on March 10, 2016, 12:01:20 am
I guess I'll look into it before release, then.

By the way, true to form, I decided to tackle the letterboxing issue now.  minisphere's window scaling/resizing logic was in need of some refactoring anyway.  main.c being the oldest file in the codebase, it's quite a mess of spaghetti code compared to the rest of the engine and could definitely use some cleanup sooner rather than later.
Title: Re: minisphere 3.0rc5
Post by: Fat Cerberus on March 11, 2016, 05:05:06 pm
minisphere 3.0rc5 is now available, with fixes for several bugs found in 3.0rc4 by DaVince, and one fairly large change that really shouldn't have been done during code freeze but oh well: I added letterboxing support.  Linux deb packages also now automatically associate Sphere file types (.sgm, .s2gm, .spk) with minisphere.
Title: Re: minisphere 3.0rc6
Post by: Fat Cerberus on March 17, 2016, 01:30:08 am
Just posted 3.0rc6, the final release candidate.  miniRT has been overhauled and is now provided in module form, loadable using require('miniRT/threads'); etc.  Full API documentation for miniRT is finally included as well, so hopefully that'll make the library more discoverable.  It's definitely worth looking into if you don't want to reinvent the wheel--it includes a lot of useful stuff like my threading library, a version of Scenario, and a text console with built-in command parser (you just have to provide the command names and implementations!).

This will be the final 3.0 release candidate.  I don't see any further changes being needed and the engine has been bug-free in my own use, but if anything else comes up I'll just make the fixes to the final release without doing more RCs.  If I don't draw the line here, my perfectionism is going to be the death of me. :P

So anyway, go download rc6 and have fun!
Title: Re: minisphere 3.0rc6
Post by: HopeMetal on March 19, 2016, 08:47:21 am
I'm running into an issue when I try to put a game into windowed mode with F10 - it stays fullscreen, but the windowed game goes in the corner without any window border and everything else is black.
Title: Re: minisphere 3.0rc6
Post by: Fat Cerberus on March 19, 2016, 09:44:25 am
What's your graphics card?  Is it an Intel iGPU?  I have this too on my laptop (Core i3-5500U), but it affects all OpenGL games, not just minisphere--I think it's a driver issue with the newer Intel chips.  So there's not a lot I can do short of putting Allegro in D3D mode (which disables shaders).
Title: Re: minisphere 3.0rc6
Post by: HopeMetal on March 19, 2016, 01:40:42 pm
It works fine when I made minishpere use my dedicated nVidia card. Thanks for the lead.
Title: Re: minisphere 3.0rc6
Post by: Fat Cerberus on March 24, 2016, 12:35:16 pm
I'm finalizing building the binaries for minisphere 3.0.0 to be released on the 28th.  If anyone has encountered any issues with the RC, now is the time to speak up.  ;)
Title: Re: minisphere 3.0rc6
Post by: Radnen on March 24, 2016, 04:33:12 pm
I've been using this version and have not ran into any serious issues. I think I got it to crash once, but I haven't been able to reproduce it... I'll run stress tests later in the week and get back to you.
Title: Re: minisphere 3.0.0
Post by: Fat Cerberus on March 27, 2016, 11:42:07 pm
minisphere 3.0 is finally out! ;D  It's worth noting that this is minisphere's birthday - version 1.0, the first version to be almost fully compatible with Sphere 1.x, was released a year ago on this day, March 28.  Such a momentous occasion warrants a momentous release, and minisphere 3.0 delivers!  New in this release: The SSJ command-line debugger, TypeScript support, a fully documented miniRT, many, many undocumented bug fixes, and official support for Ubuntu Linux!

I decided at the last minute to bundle Sphere Studio in the (Windows) release after all.  And a note to users of 3.0rc6: Download 3.0.0.  As I suspected would happen, I had to make a few last-minute bug fixes before the release.

Anyway, go download minisphere 3.0.0 and have fun!  See the OP for a list of breaking changes.

edit: I forgot to mention, minisphere is also available on Ubuntu via PPA:

To install minisphere from PPA, open Terminal and run the following commands:
Title: Re: minisphere 3.0.0
Post by: Eggbertx on March 28, 2016, 04:34:55 pm
Just built 3.0 in Linux, and with few exceptions (like it segfaulting if you run it with a nonexistent folder as the game argument) it seems to be running perfectly.
Title: Re: minisphere 3.0.0
Post by: Fat Cerberus on March 28, 2016, 04:35:56 pm
If you use spherun instead of minisphere it will run windowed by default since that's used for debugging where fullscreen is generally undesirable.  This enables the debugger and other development features so there may be a slight performance hit, but it shouldn't be significant.  Do note however that spherun is designed for command-line use and therefore requires a game path--it won't run the startup game automatically.  What I can do for 3.0.1 is have the engine save the window/fullscreen state from the last run, perhaps per-game (spherun would still default to windowed).  That wouldn't be too difficult to do.

I'll look into that segfault, thanks for testing. :)
Title: Re: minisphere 3.0.0
Post by: Eggbertx on March 28, 2016, 04:44:41 pm
Oh alright, sounds good. Also, is it safe to add system scripts (say, from 1.6) to usr/share/system, as long as they don't conflict with the files already there, to provide compatibility with games already written for 1.6?
Title: Re: minisphere 3.0.0
Post by: Fat Cerberus on March 28, 2016, 04:50:01 pm
Yeah, that should be perfectly fine to do--the only thing in there is Sphere 1.5 scripts, modern stuff like miniRT is placed elsewhere (in /usr/share/minisphere/commonjs).  Just be aware that minisphere doesn't implement the full 1.6 API so depending on what the 1.6 scripts do, some things might not work.
Title: Re: minisphere 3.0.0
Post by: Eggbertx on March 28, 2016, 04:56:16 pm
That's fine, I didn't use 1.6 much anyway. I guess I'll be using minisphere from now on, writing code with Visual Studio Code, doing images with GIMP (or the original editor), and spriteset and mapping stuff with the original editor as well.
Title: Re: minisphere 3.0.0
Post by: Fat Cerberus on March 28, 2016, 04:58:09 pm
Make sure you test out SSJ at some point too.  Sphere JS debugging = awesome. :D
Title: Re: minisphere 3.0.0
Post by: Eggbertx on March 28, 2016, 05:05:15 pm
I've never really used built-in debugging tools, and have generally just gone with printf. For my imageboard server, I wrote a wrapper for the main printf function.
Code: [Select]

printf(n int,format string,a ...interface{})

with 0 being startup and critical info/errors, 1 being warnings, and 2 being benchmarking and less important info.
Title: Re: minisphere 3.0.0
Post by: Eggbertx on March 28, 2016, 05:07:26 pm
Oh, and n is a constant set by the Makefile
Title: Re: minisphere 3.0.0
Post by: Fat Cerberus on March 28, 2016, 05:11:25 pm
You can do a lot with printf, it's true, but at some point it's just not enough.  When I was using Sphere 1.5 to develop Specs, there were so many battle engine bugs I hit that took me hours to find even with strategically-placed Abort calls that would have taken literally 5 seconds to find with a debugger.  In a simple codebase, instrumentation is enough, but once it gets complex enough a stepping debugger is very welcome.  Particularly valuable is being able to walk up the callstack and see variables at each level, which is very difficult to emulate with prints.
Title: Re: minisphere 3.0.0
Post by: Eggbertx on March 28, 2016, 05:15:02 pm
Ah, alright. I'll definitely look into that then. Is there any way to set breakpoints without Sphere Studio, or does that not matter?

Oh wow, never mind, this is pretty neat
Title: Re: minisphere 3.0.0
Post by: Fat Cerberus on March 28, 2016, 05:17:38 pm
Yeah, "bp file:line" will set a breakpoint and give you its ID number which you can use later to clear it.  Run "man ssj" on command line for a full list of commands (or just "help" in SSJ works too).
Title: Re: minisphere 3.0.0
Post by: Fat Cerberus on March 29, 2016, 01:19:16 am
@Eggbert: I fixed that segfault you mentioned.  I had done some last-minute refactoring to the display code to support letterboxing, and it seems I introduced a crash bug in the process.  It will be fixed in 3.0.1.
Title: Re: minisphere 3.0.1
Post by: Fat Cerberus on March 29, 2016, 01:56:58 pm
Well, true to form, minisphere 3.0.1 gets released a day after a major update.  I was going to hold off on this until I amassed more fixes, but the command-line segfault bugged me.  This release also adds SetLayerSize() requested (sort of :P) by Eggbert.

@DaVince: minisphere 3.0.1 is up on the PPA.  To update automatically:


I'm not sure why the first step is necessary, but if you don't do it it will just tell you you have the latest version already.
Title: Re: minisphere 3.0.1
Post by: DaVince on March 29, 2016, 05:40:47 pm

I'm not sure why the first step is necessary, but if you don't do it it will just tell you you have the latest version already.

The update command updates your local package manager so that it becomes aware of all the new available program versions. And then you actually install those updates with "upgrade". It's how the system knows when to notify you of updates (and which ones). :)
Title: Re: minisphere 3.0.1
Post by: Flying Jester on March 29, 2016, 07:13:27 pm
I once spent quite a long time wondering why FreeBSD wasn't updating their version of clang until I discovered the difference between upgrade and update :P

I'm a little surprised that Ubuntu's package manager makes the distinction when Arch's and Fedora's do not. It's a wild world we live in.
Title: Re: minisphere 3.0.1
Post by: Fat Cerberus on March 30, 2016, 12:08:49 pm
The next thing I want to work on is improving the FileStream API, which is currently very basic--it only has "read" and "write" methods that work with ArrayBuffers.  This makes it difficult to extract, e.g., strings from files (for whatever reason, DataView has no string encoding/decoding functions even though in ES5+ strings are unequivocally UTF-8).  I want to take inspiration from the Node.js Buffer object, which has reading primitives for various types like readInt16LE, readUint32BE, etc.  This will make it easier, I think, to write binary loaders in JS to handle Sphere file types which I've found to be quite tedious to do when working with raw ArrayBuffers.

The one thing I'm wondering about is whether it would be clearer to have separate 'LE' and 'BE' call variants like Node, or do what DataView does and take an optional argument for integer-reading functions specifying the endianness (default is BE).
Title: Re: minisphere 3.0.1
Post by: Fat Cerberus on March 30, 2016, 11:00:51 pm
Following up on the above, this is the API I went with:


There are no separate methods for int16, int32, etc.  Instead you have read/writeInt() and read/WriteUInt(), which take an argument specifying the number of bytes (1-6) used to encode the value.  I got this idea from the Node.js Buffer API, which is actually quite a bit more powerful--or at least, less confusing--than the ArrayBuffer/TypedArray/DataView mess in ES6.  In fact I actually adapted the variable-length integer encoding/decoding code from the Node.js source.  It rather surprised me that Node is MIT-licensed (which allowed me to borrow code)--I was expecting LGPL or similar.

The full list of FileStream read methods is thus (with a matching set of write methods):


So if you wanted to read a 16-bit little-endian integer from a file, you'd do:
Code: (javascript) [Select]

/* readInt(num_bytes[, little_endian = false]); */
var num_pigs = file.readInt(2, true);  // 16-bit = 2 bytes
Title: Re: minisphere 3.0.1
Post by: Eggbertx on March 31, 2016, 12:25:54 am

I'm a little surprised that Ubuntu's package manager makes the distinction when Arch's and Fedora's do not. It's a wild world we live in.

When you want to update, you update the whole local database by running
Code: [Select]
# pacman -Sy

Which syncs the database
And then
Code: [Select]
# pacman -Syu

Which updates. Or you can just do the bottom one and it'll check to make sure that the local database is synced.
Title: Re: minisphere 3.0.1
Post by: Flying Jester on March 31, 2016, 12:28:40 am
@Eggbert: I haven't used Arch in some time, so I probably just forgot about that.

@Lord English: Node's license is because Joyent is heavily invested in using it in proprietary systems based on OpenSolaris, solely on amd64 systems. It's a weird and wild world out there.

Hopefully someday they get together with Fujitsu and save the remnants of Sun's empire. But I know they never will.
Title: Re: minisphere 3.0.1
Post by: Fat Cerberus on March 31, 2016, 12:44:59 am
It would probably be a good idea to also add the typed read and write methods to IOSocket.  Raw ArrayBuffers and ByteArrays are actually quite tedious to work with.
Title: Re: minisphere 3.0.1
Post by: Eggbertx on March 31, 2016, 02:49:40 pm
Just for the record, if anyone uses Arch, it's uploaded to the AUR here (https://aur.archlinux.org/packages/minisphere/).
Title: Re: minisphere 3.0.1
Post by: Eggbertx on March 31, 2016, 06:54:38 pm
How do I have the game start without showing the FPS in the bottom right? Also, in the past, by default, didn't pressing the escape key exit the map engine, unless you set something to block it?
Title: Re: minisphere 3.0.1
Post by: Fat Cerberus on March 31, 2016, 06:59:14 pm
"spherun" shows the FPS by default because it's meant for development, "minisphere" will hide it by default but you will have to use the "--window" option to not start in fullscreen.

I don't ever remember ESC exiting the map engine by default though, that would be undesirable for most games and be a pain in the neck to have to work around.  You'll have to add your own code to do that in your game if you want it.
Title: Re: minisphere 3.0.1
Post by: Eggbertx on March 31, 2016, 07:03:44 pm

"spherun" shows the FPS by default because it's meant for development, "minisphere" will hide it by default but you will have to use the "--window" option to not start in fullscreen.

Oh, right, that makes sense.

Quote

I don't ever remember ESC exiting the map engine by default though, that would be undesirable for most games and be a pain in the neck to have to work around.  You'll have to add your own code to do that in your game if you want it.

Yeah, 1.5 and I think 1.6 did. Adding in some code to quit if IsKeyPressed(KEY_ESCAPE) would be pretty trivial on the scripting end.
Title: Re: minisphere 3.0.1
Post by: Fat Cerberus on March 31, 2016, 07:04:23 pm
By the way, why is that showing "61 fps" instead of "61/61 fps"?  Are you using your own throttling code?  You'd do better to call SetFrameRate() and let the engine manage it, to avoid wasting CPU cycles.
Title: Re: minisphere 3.0.1
Post by: Eggbertx on March 31, 2016, 07:17:38 pm
It shows only 61 if you pass just one parameter (the location of the map file) because I thought I remember reading something about minisphere not requiring the framerate. If I do MapEngine("mapfile.rmp", FPS), it shows the 61/61. I haven't written any code other than CreatePerson,AttachInput,AttachCamera, and MapEngine. The usual.
Title: Re: minisphere 3.0.1
Post by: Fat Cerberus on March 31, 2016, 07:23:12 pm
Yeah, MapEngine() FPS is not required, but I believe it defaults to the global framerate (which itself defaults to 0, i.e. unthrottled).  So if you call SetFrameRate() at any point and then MapEngine(), you will get that framerate by default.  You should never run unthrottled in a production game because if vsync is turned off in the driver config, you could potentially do 1000+ updates per second! :P
Title: Re: minisphere 3.0.1
Post by: Eggbertx on March 31, 2016, 07:25:03 pm
Alright then. So is doing MapEngine(MAPFILE,FPS) ideal then?
Title: Re: minisphere 3.0.1
Post by: Fat Cerberus on March 31, 2016, 07:26:56 pm
Ideally I would call SetFrameRate(FPS) at game startup and then call MapEngine without it.  This way if you do any FlipScreen loops, the framerate will be in sync with the map engine (minisphere automatically throttles FlipScreen for you)
Title: Re: minisphere 3.0.1
Post by: Eggbertx on March 31, 2016, 07:28:14 pm
Alright. I'll keep that in mind from now on.
Title: Re: minisphere 3.0.1
Post by: DaVince on March 31, 2016, 07:32:12 pm
Pressing escape to quit the whole game while in the map engine WAS a thing in regular Sphere, but I'm honestly glad you're not including that functionality in minisphere. It was, as you said, unintuitive.
Title: Re: minisphere 3.0.1
Post by: Fat Cerberus on March 31, 2016, 10:18:46 pm
Huh, you're right - I'm surprised I never knew this, seems like something I would have hit by accident (and been very annoyed).  It also appears that F1 disables throttling, which might be useful for benchmarking but that's about it.  Even during development I can't see a reason to do this, other than maybe as an ad-hoc fast-forward feature--which wouldn't even work in the case of vsync or laggy sections.  Here's the relevant part of the Sphere 1.x source:

https://github.com/sphere-group/sphere/blob/master/sphere/source/engine/map_engine.cpp#L5855-L5873

I stand by minisphere not supporting it.  Reading the 1.x code, it appears to only way to override it is to call BindKey(), which a beginner might not even think to do since they'd just think it's a hardcoded "system" key.  If you need it, it's easy enough to add a check in an update script or miniRT thread. :)
Title: Re: minisphere 3.0.1
Post by: Radnen on March 31, 2016, 10:32:35 pm

I stand by minisphere not supporting it.  Reading the 1.x code, it appears to only way to override it is to call BindKey(), which a beginner might not even think to do since they'd just think it's a hardcoded "system" key.  If you need it, it's easy enough to add a check in an update script or miniRT thread. :)


Don't worry if it doesn't help support backwards compatibility. All games I've played had more proper ways of quitting the game. :P there's always ALT+F4.
Title: Re: minisphere 3.0.2
Post by: Fat Cerberus on April 01, 2016, 03:17:08 am
I found a nasty bug in the Sphere Studio debugger (F10 to Step Over didn't work and performed Step Out instead--ouch!), so I uploaded a new release.  As a bonus you also get the new FileStream methods I mentioned earlier to read and write numbers and strings.  I was going to hold that back until 3.1 (which is a ways off), but I fast-tracked that bug fix and didn't feel like moving it onto a separate branch, so now everyone gets the feature early. ;)
Title: Re: minisphere 3.0.2
Post by: Fat Cerberus on April 01, 2016, 02:43:28 pm
I'm currently working on a miniRT module which will make reading data from binary files in JS much easier, by reading data items into an object based on a JSON schema.  For example, this is a schema to read an RMP map file header:

Code: (javascript) [Select]
[
{ id: 'magic', type: 'fstring', size: 4 },
{ id: 'version', type: 'uintLE', size: 2 },
{ id: 'type', type: 'uintLE', size: 1 },
{ id: 'numLayers', type: 'uintLE', size: 1 },
{ type: 'raw', size: 1 },  // reserved
{ id: 'numEntities', type: 'uintLE', size: 2 },
{ id: 'startX', type: 'uintLE', size: 2 },
{ id: 'startY', type: 'uintLE', size: 2 },
{ id: 'startLayer', type: 'uintLE', size: 1 },
{ id: 'startDir', type: 'uintLE', size: 1 },
{ id: 'numStrings', type: 'uintLE', size: 2 },
{ id: 'numZones', type: 'uintLE', size: 2 },
{ id: 'repeating', type: 'bool' },
{ type: 'raw', size: 234 },  // reserved
{ id: 'strings', type: 'array', count: '@numStrings', subtype: { type: 'pstringLE', size: 2 } },
];


I got this idea from looking at the Turbo runtime. :)
Title: Re: minisphere 3.0.2
Post by: Flying Jester on April 01, 2016, 04:30:23 pm
Yes, I noticed you were running into the same problems I was with how tedious reading binary data from files is in script :)

I still think it's better to layer this on top of ArrayBuffers than to change the underlying data reading API, though. That way you only need a single native method for each type of object, and the same script methods can operate on, say, a file, a socket, or any ArrayBuffer that the developer wants to use, too.
Title: Re: minisphere 3.0.2
Post by: Fat Cerberus on April 01, 2016, 04:39:44 pm
The main problem I was running into with ArrayBuffers is that there's no built-in JS facility for decoding strings from them.  DataView gives you plenty of stuff for decoding integers and floats, but nothing for strings, so you end up having to construct them character-by-character (possibly including UTF-8 decoding logic) by reading single bytes at a time.  I don't trust Duktape's performance quite enough to do this in script--plus you run into performance issues with string interning when building a string piecemeal like that--so it was easier just to put the string I/O into the engine.

Reading the Turbo code, I saw you had the same problem, and ended up falling back on CreateStringFromByteArray().  Since I consider the ByteArray stuff deprecated at this point, I didn't want to do that.

By the way, here's a schema to read an RMP file through the layer data.  Alas, I don't think I can represent the entire RMP format using a single read call unless I make the schema system Turing complete--persons and triggers have a completely different set of fields, but are both interleaved within the "entities" section.

Code: (javascript) [Select]

var segmentSchema =
[
{ id: 'x1', type: 'uintLE', size: 4 },
{ id: 'y1', type: 'uintLE', size: 4 },
{ id: 'x2', type: 'uintLE', size: 4 },
{ id: 'y2', type: 'uintLE', size: 4 },
];

var layerSchema =
[
{ id: 'width', type: 'uintLE', size: 2 },
{ id: 'height', type: 'uintLE', size: 2 },
{ id: 'flags', type: 'uintLE', size: 2 },
{ id: 'parallaxX', type: 'floatLE' },
{ id: 'parallaxY', type: 'floatLE' },
{ id: 'scrollX', type: 'floatLE' },
{ id: 'scrollY', type: 'floatLE' },
{ id: 'numSegments', type: 'uintLE', size: 4 },
{ id: 'reflective', type: 'bool' },
{ type: 'reserved', size: 3 },
{ id: 'name', type: 'pstringLE', size: 2 },
{ id: 'tilemap', type: 'array', count: '@height', subtype: { type: 'array', count: '@width', subtype: { type: 'uintLE', size: 2 } } },
{ id: 'segments', type: 'array', count: '@numSegments', subtype: { type: 'object', schema: segmentSchema } },
];

var mapSchema =
[
{ id: 'signature', type: 'fstring', size: 4, regex: "\\.rmp" },
{ id: 'version', type: 'uintLE', size: 2 },
{ id: 'type', type: 'uintLE', size: 1 },
{ id: 'numLayers', type: 'uintLE', size: 1 },
{ type: 'reserved', size: 1 },  // reserved
{ id: 'numEntities', type: 'uintLE', size: 2 },
{ id: 'startX', type: 'uintLE', size: 2 },
{ id: 'startY', type: 'uintLE', size: 2 },
{ id: 'startLayer', type: 'uintLE', size: 1 },
{ id: 'startDir', type: 'uintLE', size: 1 },
{ id: 'numStrings', type: 'uintLE', size: 2 },
{ id: 'numZones', type: 'uintLE', size: 2 },
{ id: 'repeating', type: 'bool' },
{ type: 'reserved', size: 234 },  // reserved
{ id: 'strings', type: 'array', count: '@numStrings', subtype: { type: 'pstringLE', size: 2 } },
{ id: 'layers', type: 'array', count: '@numLayers', subtype: { type: 'object', schema: layerSchema } },
];
Title: Re: minisphere 3.0.2
Post by: Flying Jester on April 01, 2016, 04:50:46 pm

Reading the Turbo code, I saw you had the same problem, and ended up falling back on CreateStringFromByteArray().  Since I consider the ByteArray stuff deprecated at this point, I didn't want to do that.


I did not fall back on it in that sense. There is no engine-supplied CreateStringFromByteArray in TurboSphere. It is a script function that takes any iterable object, and constructs a string using the contents as codepoints. It has the name "CreateStringFromByteArray" solely because it is compatible with that function from Sphere 1.5

Regarding the more complex schema, I decided I wanted very simple schema, and it was OK to add a little bit of format-specific branching for each reader. This was also quite important for supporting all the RSS Spriteset versions.
Title: Re: minisphere 3.0.2
Post by: Fat Cerberus on April 01, 2016, 05:00:31 pm
One problem with that approach is that it doesn't handle multi-byte character sets like UTF-8 well - if you just blindly build up a JS string from individual bytes you'll end up with a different string than if you decoded it as UTF-8.  And there's the string interning issue I mentioned also.

Node.js solves all of this nicely - its Buffer object has a toString() function which actually encodes the buffer contents as a string.  ArrayBuffer.toString() just uselessly returns "[object ArrayBuffer]". :-\
Title: Re: minisphere 3.0.2
Post by: Flying Jester on April 01, 2016, 05:02:53 pm
Well, it could handle UTF-8. I just never got around to adding that.
Title: Re: minisphere 3.0.2
Post by: Fat Cerberus on April 02, 2016, 02:59:14 am
I did it - I got miniRT to be able to load a complete RMP map file with a single read call!  See attached screenshot.  The trick was to add a simple conditional mechanism to the schema, like so:

Code: (javascript) [Select]

var entitySchema =
[
{ type: 'uintLE', size: 2, id: 'x' },
{ type: 'uintLE', size: 2, id: 'y' },
{ type: 'uintLE', size: 2, id: 'layer' },
{ type: 'uintLE', size: 2, id: 'type' },
{ type: 'reserved', size: 8 },
{ type: 'switch', field: 'type', id: 'data', cases: [
{ value: 1, type: 'object', schema: personSchema },
{ value: 2, type: 'object', schema: triggerSchema },
]},
];


...then proceeded to spend almost two hours in the JS debugger (I am *so* glad I made that thing!) trying to debug why I couldn't load the file despite all the schemas being correct and double- and triple-checking them, but FileStream.read() seemed to be reading too many bytes anyway.  Come to find out that dumb ol' me had opened the file in text mode so that whenever it encountered a CRLF pair it would convert it to LF (thus reading 2 bytes but only counting it as 1).  DOH! :-X

And yes, the Vegeta wallpaper is entirely appropriate considering how much time I spent in SSJ2. :P
Title: Re: minisphere 3.0.2
Post by: Fat Cerberus on April 03, 2016, 01:41:30 am
New feature in the latest build: XML parsing!  I found this lightweight XML parser for JavaScript yesterday:
https://github.com/isaacs/sax-js

It was made with Node.js in mind but thanks to me implementing the CommonJS module system in minisphere, it runs just fine there too. :)  However because Node is async, the event-based streaming API the XML library exposes isn't really a good match for the Sphere architecture... so I added an easy-to-use wrapper for it as part of miniRT.  That lets you skip all the event listener setup rigamarole and just do this:

Code: (javascript) [Select]

var xml = require('miniRT/xml');
// ...fetch some XML from a file or something...
var dom = xml.parse(xmlText);


Which makes decoding XML as easy as JSON! :D

There's an ulterior motive, of course.  I eventually want to support the Tiled map editor, which uses XML as its default save format.
Title: Re: minisphere 3.0.2
Post by: Fat Cerberus on April 04, 2016, 02:10:12 am
As another stepping stone in eventually making a JS-based map engine, I've added the ability to construct an image from an ArrayBuffer, which is needed to allow tilesets and such to be loaded in script.  miniRT will also be aware of this functionality, so it can be used in schemas, like this one here to load RTS tilesets:

Code: (javascript) [Select]
/**
*  Sphere 1.x tileset format schema CommonJS module
*  a schema for reading Sphere 1.x tileset files using miniRT/binary
*  (c) 2015-2016 Fat Cerberus
**/

module.exports =
[
{ id: 'signature', type: 'fstring', size: 4, regex: "\\.rts" },
{ id: 'version', type: 'uintLE', size: 2, range: [ 1, 1 ] },
{ id: 'numTiles', type: 'uintLE', size: 2 },
{ id: 'tileWidth', type: 'uintLE', size: 2 },
{ id: 'tileHeight', type: 'uintLE', size: 2 },
{ id: 'bpp', type: 'uintLE', size: 2, values: [ 32 ] },
{ id: 'compressed', type: 'bool', values: [ false ] },
{ id: 'hasObstructions', type: 'bool' },
{ type: 'reserved', size: 240 },
{ id: 'images', type: 'array', count: '@numTiles', subtype: { type: 'image', width: '@tileWidth', height: '@tileHeight' } },
{ id: 'tiles', type: 'array', count: '@numTiles', subtype: { type: 'object', schema: [
{ type: 'reserved', size: 1 },
{ id: 'animated', type: 'bool' },
{ id: 'nextTile', type: 'uintLE', size: 2 },
{ id: 'delay', type: 'uintLE', size: 2 },
{ type: 'reserved', size: 1 },
{ id: 'obstructionType', type: 'uintLE', size: 1 },
{ id: 'numSegments', type: 'uintLE', size: 2 },
{ id: 'nameLength', type: 'uintLE', size: 2 },
{ id: 'terraformed', type: 'bool' },
{ type: 'reserved', size: 19 },
{ id: 'name', type: 'fstring', size: '@nameLength' },
{ id: 'obsmap', type: 'switch', field: 'obstructionType', cases: [
{ value: 0, type: 'reserved', size: 0 },
{ value: 2, type: 'array', count: '@numSegments', subtype: { type: 'object', schema: [
{ id: 'x1', type: 'uintLE', size: 2 },
{ id: 'y1', type: 'uintLE', size: 2 },
{ id: 'x2', type: 'uintLE', size: 2 },
{ id: 'y2', type: 'uintLE', size: 2 },
]}},
]},
]}},
];


The Image() constructor takes width, height, and an ArrayBuffer as arguments and populates the image with the pixels in the buffer.  The buffer must be large enough to represent the entire image, and the pixels are assumed to be in 32-bit RGBA order, which seems idiomatic for Sphere.  Here's an example of using the feature from my WIP binary file reader:

Code: (javascript) [Select]

case 'image':
var width = _fixup(fieldType.width, data);
var height = _fixup(fieldType.height, data);
if (typeof width != 'number' || typeof height != 'number')
throw new Error("missing or invalid width/height for `image` field");
var pixelData = stream.read(width * height * 4);
value = new Image(width, height, pixelData);
break;


This also, as a bonus, would allow you to do software surfaces in script (in minisphere, CreateSurface() makes hardware surfaces)--although I'm not sure what the performance of that would be.  In any case it's something FJ mentioned I should have early on in the project's life, and now I finally implemented it. ;)
Title: Re: minisphere 3.0.2
Post by: Flying Jester on April 04, 2016, 02:55:18 am
Loading Surfaces straight from ArrayBuffers was also something I needed/wanted for the Turbo map engine. I wouldn't say it's strictly necessary, but it made loading tilesets, fonts, and spritesets MUCH faster.

I worried a little about the performance as well. Regardless, I think the solution that would work best would be to expose a memcpy API to JS, to allow copying between ArrayBuffers. In fact, it would be useful to have versions of many C functions like memcpy, memset, etc. in JS, and it would also be easy to make script equivalents to allow any scripts that used them to work on systems without native functionality.

It's notable that SDL2's blit is actually faster than memcpy'ing, since it can make use of RLE data to skip unnecessary reads, but this would still give you performance quite close to a native implementation.

That's also a part of why I'm going in much farther on ArrayBuffers in Turbo. I think they are a really useful thing for JS, they just need some really basic features added on top (like a string conversion, and memcpy) to really be used most effectively. Unfortunately, given the philosophy of the browser developers, these things will likely stay out of JS proper for the forseeable future.
Title: Re: minisphere 3.0.2
Post by: Fat Cerberus on April 04, 2016, 03:19:40 am
I definitely agree, I've found that using only stock ArrayBuffers isn't really adequate for most of the use cases they are ostensibly meant to solve, you need to layer some extra stuff on top of them first.  Part of the issue is that an ArrayBuffer object by itself is pretty much a black box, like a void pointer that you can't cast.  You can construct a typed array or DataView on top of it to manipulate its contents, but copying data around in bulk between buffers like with memcpy in C?  Yeah, not going to happen.  And it's also rare that you see an ArrayBuffer used as input, from what I've seen they are typically created on-demand whenever some data is generated (I'm guilty of this too, FileStream.read() returns a new one for each call...), which honestly defeats the purpose.

I'll take that memcpy API idea into consideration as minisphere 3.1 develops.
Title: Re: minisphere 3.0.2
Post by: Fat Cerberus on April 04, 2016, 03:28:39 am
By the way the miniRT binary file loader is turning out to be one of the best ideas I ever had.  Loading a tileset:

Code: (javascript) [Select]

var binary = require('miniRT/binary');
var schema = require('miniRT/schema/tileset');
var tileset = binary.load('maps/TestvilleTiles.rts', schema);
while (true) {
tileset.images[1].blit(0, 0);
FlipScreen();
}


And to load a map instead, it's as easy as substituting the correct schema:
Code: (javascript) [Select]

var schema = require('miniRT/schema/map');
var map = binary.load('maps/Testville.rts', schema);


The tricky part is to make the schema format flexible enough to represent entire file formats while still remaining JSON serializable.  So far this hasn't been an issue, but we'll see what happens when I get to the spriteset loader (.rss).  I still remember how hard that was to get right when I implemented it in C, so much so that it managed to earn itself a "HERE BE DRAGONS!" comment.  Which is NOT a compliment--the only other function in the codebase that managed to earn that honor was main() (I'm slowly cleaning it up though). :P
Title: Re: minisphere 3.0.2
Post by: Flying Jester on April 04, 2016, 05:44:29 am

I'll take that memcpy API idea into consideration as minisphere 3.1 develops.


I should clarify, I really meant memcpy on TypedArrays rather than directly on raw ArrayBuffers, and I expect such an API to operate on the element sizes, not always byte sizes. But given how both SM and V8 work, it would result in a small amount of arithmetic and then a call to memcpy uner the hood.
Title: Re: minisphere 3.0.2
Post by: DaVince on April 04, 2016, 08:41:33 am

Code: (javascript) [Select]

var xml = require('miniRT/xml');
// ...fetch some XML from a file or something...
var dom = xml.parse(xmlText);


Which makes decoding XML as easy as JSON! :D

There's an ulterior motive, of course.  I eventually want to support the Tiled map editor, which uses XML as its default save format.

Tiled? Yesss! That's something I had eventually hoped for, but didn't ask about anymore because XML can be a bitch to parse. This is excellent news and would bring Sphere that much closer to being cross-platform friendly. :)
Title: Re: minisphere 3.0.2
Post by: Fat Cerberus on April 04, 2016, 01:03:42 pm

I should clarify, I really meant memcpy on TypedArrays rather than directly on raw ArrayBuffers, and I expect such an API to operate on the element sizes, not always byte sizes. But given how both SM and V8 work, it would result in a small amount of arithmetic and then a call to memcpy uner the hood.


That makes sense, but here's something interesting: In Duktape's implementation of ArrayBuffer, they are secretly byte arrays - you can index them using [] without constructing a Uint8Array on top of them.  I tend to avoid taking advantage of it though because it's not standards compliant.
Title: Re: minisphere 3.0.2
Post by: Fat Cerberus on April 05, 2016, 12:43:27 am
I did it - I got miniRT to load RSS spritesets in one pass (any version).  I thought it was impossible, but I just needed to make a small change to the schema handling and it all came together.

The only tweak I needed to make was, when processing a sub-schema (i.e. an 'object' field), to construct the sub-object using the parent object as its prototype.  This allows the individual frame headers to reference properties from the main RSS header, and everything works.

Code: (javascript) [Select]

/**
*  Sphere 1.x tileset format schema CommonJS module
*  a schema for reading Sphere 1.x tileset files using miniRT/binary
*  (c) 2015-2016 Fat Cerberus
**/

module.exports =
[
{ id: 'signature', type: 'fstring', size: 4, regex: "\\.rss" },
{ id: 'version', type: 'uintLE', size: 2, range: [ 1, 3 ] },
{ id: 'numImages', type: 'uintLE', size: 2 },
{ id: 'frameWidth', type: 'uintLE', size: 2 },
{ id: 'frameHeight', type: 'uintLE', size: 2 },
{ id: 'numPoses', type: 'uintLE', size: 2 },
{ id: 'base', type: 'object', schema: [
{ id: 'x1', type: 'uintLE', size: 2 },
{ id: 'y1', type: 'uintLE', size: 2 },
{ id: 'x2', type: 'uintLE', size: 2 },
{ id: 'y2', type: 'uintLE', size: 2 },
]},
{ type: 'reserved', size: 106 },
{ id: 'data', type: 'switch', field: 'version', cases: [
{ value: 1, type: 'object', schema: [
{ id: 'images', type: 'array', count: 64, subtype: { type: 'image', width: '@frameWidth', height: '@frameHeight' } },
]},
{ value: 2, type: 'object', schema: [
{ id: 'poses', type: 'array', count: '@numPoses', subtype: { type: 'object', schema: [
{ id: 'numFrames', type: 'uintLE', size: 2 },
{ type: 'reserved', size: 62 },
{ id: 'frames', type: 'array', count: '@numFrames', subtype: { type: 'object', schema: [
{ id: 'width', type: 'uintLE', size: 2 },
{ id: 'height', type: 'uintLE', size: 2 },
{ id: 'delay', type: 'uintLE', size: 2 },
{ type: 'reserved', size: 26 },
{ id: 'image', type: 'image', width: '@width', height: '@height' },
]}},
]}},
]},
{ value: 3, type: 'object', schema: [
{ id: 'images', type: 'array', count: '@numImages', subtype: { type: 'image', width: '@frameWidth', height: '@frameHeight' } },
{ id: 'poses', type: 'array', count: '@numPoses', subtype: { type: 'object', schema: [
{ id: 'numFrames', type: 'uintLE', size: 2 },
{ type: 'reserved', size: 6 },
{ id: 'name', type: 'pstringLE', size: 2 },
{ id: 'frames', type: 'array', count: '@numFrames', subtype: { type: 'object', schema: [
{ id: 'imageIndex', type: 'uintLE', size: 2 },
{ id: 'delay', type: 'uintLE', size: 2 },
{ type: 'reserved', size: 4 },
]}},
]}},
]},
]},
];


Having written these schemas, it occurs to me that the Sphere 1.x file formats are ridiculously complex.  They definitely suffer from backwards-compatibility baggage, especially the spriteset format (that said, version 3 isn't terrible, but it's not really optimal either).  I'm thinking it might be high time we deprecated them and replaced with something a bit more modern and easier to work with.  For starters, a more modular format would be welcome.  JSON, perhaps?
Title: Re: minisphere 3.0.2
Post by: Flying Jester on April 05, 2016, 02:16:11 am
Well, I don't think using a single schema to represent the RSS format is a good idea, unless you only plan on supporting V3.

I actually rather like some of the formats, in particular the font format is nice and to the point. But I'll agree, I think that RTS, RSS, RMP are a little long in the tooth.
Title: Re: minisphere 3.0.2
Post by: Fat Cerberus on April 05, 2016, 02:19:51 am

Well, I don't think using a single schema to represent the RSS format is a good idea, unless you only plan on supporting V3.


Nonetheless, I managed to pull it off. :P  I posted the full schema I used above.  I used a bit of conditional magic, switching on the version field to read different structures depending on which version number is in the header.  There's also some basic validation built in too, as you can see on the version and signature fields.
Title: Re: minisphere 3.0.2
Post by: Fat Cerberus on April 05, 2016, 03:10:20 am
On an unrelated note, I'm currently experimenting with rendering maps using the Galileo API instead of legacy blitting.  I already make an atlas for tilesets on load, so the entire map could be represented as a single Shape and thus rendered with a single GPU call, with the vertices cached on the GPU (Allegro 5.2 supports vertex buffers).

The one thing I'm having trouble with is figuring out the vertex order.  I know I have to make a triangle strip, but I'm unsure how to set up the vertices and texture UV to keep the map as one continuous strip.  @FJ, you wrote the Turbo map engine to take advantage of Galileo didn't you?  Any tips you could offer?
Title: Re: minisphere 3.0.2
Post by: Flying Jester on April 05, 2016, 03:36:32 am
By the time I wrote the Turbo runtime, all TurboSphere had for graphics was Galileo :)

I had to issue an extra vertex for each tile to reset the positioning.

https://github.com/FlyingJester/TurboSphere/blob/master/bin/system/scripts/turbo/map.js#L200 (https://github.com/FlyingJester/TurboSphere/blob/master/bin/system/scripts/turbo/map.js#L200)

(At least I think that that's the line.)

I never found a better way around this, but it's not like this is an unusual problem with OpenGL-style rendering. NVidia even created an extension just to deal with this kind of thing, called the Primitive Reset, which would take the place of this extra vertex. In more complex scenes that would be more desirable, but in this case it really makes no difference compared with burning an extra vertex here.
Title: Re: minisphere 3.0.2
Post by: Fat Cerberus on April 05, 2016, 11:23:07 am
I think I understand the theory at least, you throw in an extra vertex in order to get the texturing to work properly.  What I DON'T understand is:

* Why are there 4 vertices per tile?  My understanding was that a tri-strip only requires 2+n vertices (where 'n' is the number of triangles).  A tile is two triangles, so after the first tile, you should only need 2 vertices for each one.
* Why does the extra vertex use tex_coords[3] of the next tile?  Shouldn't that be tex_coords[0]?
Title: Re: minisphere 3.0.2
Post by: Flying Jester on April 05, 2016, 12:58:12 pm
It will only need 2+n when you are not trying to use non-adjacent points in the texture on adjacent triangles like we need to. If you find a way around this, I'd love to know, but I'm pretty sure this is just how it is.

As far as the texture coordinates of the burned vertex, I think it does not matter since the triangle that the burned vertex appears in is totally flat across the bottom of the tile. The next triangle is formed up the right-hand side of the triangle (again with zero width), and then finally the first triangle of the new tile is draw. I think (it's been a while since I looked at this code) that the only non-zero-width triangle this burned vertex appears in is overdrawn, and it only appears in a single non-zero-width triangle (which spans the bottom left of the new tile, the top left, and then the top right, and is immediately overdrawn by issuing the third vertex of the new tile).

I would also like to note that it was while writing this code that I changed the default drawing mode of Galileo Shapes at or above 4 vertices to be triangle strips instead of fans. This is precisely the situation I said could not be done with a single Shape using triangle fans.
Title: Re: minisphere 3.0.2
Post by: Fat Cerberus on April 05, 2016, 01:07:06 pm
Yeah, I default to strips too for 5+ vertices.  Specifically, my defaults are:
Code: (c) [Select]

draw_mode = shape->num_vertices == 1 ? ALLEGRO_PRIM_POINT_LIST
: shape->num_vertices == 2 ? ALLEGRO_PRIM_LINE_LIST
: shape->num_vertices == 4 ? ALLEGRO_PRIM_TRIANGLE_FAN
: ALLEGRO_PRIM_TRIANGLE_STRIP;


I use a fan for == 4 vertices so that the vertices for an isolated quad (the common case for emulating a blit) can be specified clockwise, which is more intuitive than the zigzag pattern needed for a strip.  It's worth noting though that minisphere's Shape() constructor has an additional optional parameter which allows you to specify the primitive type.  That allows Galileo to also replace legacy APIs such as LineSeries(), PointSeries(), etc.
Title: Re: minisphere 3.0.3
Post by: Fat Cerberus on April 05, 2016, 02:30:30 pm
Speaking of Galileo, I found a bug in GetDefaultShaderProgram() which caused the default shaders to fail to compile.  3.0.3 fixes the issue.
Title: Re: minisphere 3.0.3
Post by: Fat Cerberus on April 06, 2016, 02:39:46 am
Initial experiments with Galileo-based map rendering is promising--running at 1920x1080, filling the screen with tiles uses around 2.5% CPU on my i7-6700HQ at 60fps when drawing tile-by-tile (= 4 vertices and 1 draw per tile), while rendering with Galileo (~5 vertices per tile, 1 draw per layer) dropped that to around 1.8%.  Now I'm curious what the results will be on a slower machine.

Now I just have to fix it so it supports repeating maps...

edit: Yay! Repeating maps work!  Hm... in theory I could add the persons on a layer to the layer's Group too... although then I'd be recalculating the entire map every frame, which defeats the purpose.
Title: Re: minisphere 3.0.2
Post by: Fat Cerberus on April 06, 2016, 10:45:00 am

As far as the texture coordinates of the burned vertex, I think it does not matter since the triangle that the burned vertex appears in is totally flat across the bottom of the tile. The next triangle is formed up the right-hand side of the triangle (again with zero width), and then finally the first triangle of the new tile is draw. I think (it's been a while since I looked at this code) that the only non-zero-width triangle this burned vertex appears in is overdrawn, and it only appears in a single non-zero-width triangle (which spans the bottom left of the new tile, the top left, and then the top right, and is immediately overdrawn by issuing the third vertex of the new tile).


It turns out that it actually does matter!  For solid tiles there's no issue, however in case of transparency you want to use (u1,v1) instead of (u2,v2) for that extra vertex.  Compare the screenshots below.  The (u1,v1) version still has a small artifact on the flame that I haven't figured out the cause of, but is otherwise perfect.  The (u2,v2) version has a very striking artifact on tiles with transparency due to "jumping" from one corner of the tile texture to the other during rendering.  That triangle isn't fully overdrawn because the tile is transparent, so the illusion is broken.

As for reducing the number of vertices, it might actually be possible.  Rather than rendering exclusively left-to-right like in the Turbo runtime, I suspect that if you construct the strip in a zigzag pattern, alternating between left-to-right and right-to-left for each line and burning a vertex for each tile to reset the texture coordinates, you could get it down to 3 vertices per tile.  This would make the rendering calculations more complicated, of course.
Title: Re: minisphere 3.0.3
Post by: Fat Cerberus on April 06, 2016, 12:22:36 pm
Hm, ignore that 3-vertices-per-tile idea.  I just realized the issue: Once you issue the burner vertex the next triangle is zero-width by definition since two of its vertices will share the same position.  A more elegant way to do this would probably be to use a triangle list, but that bumps us up to 6 vertices per tile which might be a performance issue unless you also use an index buffer.

Who said trigonometry was useless? ;)
Title: Re: minisphere 3.0.3
Post by: Flying Jester on April 06, 2016, 12:53:34 pm
Well, the "solution" is to use a primitive reset object, or so most OpenGL folk have decided. But in this case, and in fact I'm fairly certain in all 2D cases, it's less overhead and is a simpler API to just burn a vertex (whereas you may need to burn up to three vertices per triangle in 3D).

I think the other solution could be to use the UV of the next vertex. This would still have overdrawn areas, but they'd be the right texturing at least.
Title: Re: minisphere 3.0.3
Post by: Fat Cerberus on April 06, 2016, 01:03:53 pm
I will say that experimenting with Galileo I find it to be less complicated to use than I originally thought.

There are things that bug me about the JS API for Galileo though, I want to look into maybe redesigning it a bit in the next release.  I'm not sure exactly what it is but it feels awkward to use at times.  Shapes and Groups being immutable after construction is a bit annoying, for example.
Title: Re: minisphere 3.0.3
Post by: Flying Jester on April 06, 2016, 02:02:00 pm
Groups and Shapes are not immutable...or at least they aren't in TurboSphere. Groups expose a `shapes` member which holds the array of shapes. Shapes have mutable images, although vertices are immutable simply because Shapes are relatively cheap to create, and due to how OpenGL is tooled it is the same cost (or even cheaper) to just create new Shapes than to modify an existing Shape's vertices.

I probably never updated any of the specs since actually updating a Group's array of Shapes required fully setting the shapes member, since I never quite figured out how to set data accessors on pure arrays in SM.
Title: Re: minisphere 3.0.3
Post by: Fat Cerberus on April 06, 2016, 02:35:51 pm
Hm, so fixing all the glitches with transparent tiles ended up needing to burn two vertices:

Code: (c) [Select]

s_layer_groups[i] = new_group(get_default_shader());
layer_shape = new_shape(SHAPE_TRIANGLE_STRIP, get_atlas_image(tile_atlas));
for (y = 0; y < s_map->height; ++y) for (x = 0; x < s_map->width; ++x) {
tile = s_map->layers[i].tilemap[x + y * s_map->layers[i].width];
uv = get_atlas_uv(tile_atlas, tile.tile_index);
x1 = x * tile_width; x2 = x1 + tile_width;
y1 = y * tile_height; y2 = y1 + tile_height;
add_shape_vertex(layer_shape, vertex(x1, y1, uv.x1, uv.y1, rgba(255, 255, 255, 255)));
add_shape_vertex(layer_shape, vertex(x2, y1, uv.x2, uv.y1, rgba(255, 255, 255, 255)));
add_shape_vertex(layer_shape, vertex(x1, y2, uv.x1, uv.y2, rgba(255, 255, 255, 255)));
add_shape_vertex(layer_shape, vertex(x2, y2, uv.x2, uv.y2, rgba(255, 255, 255, 255)));
if (x < s_map->layers[i].width - 1) {
tile = s_map->layers[i].tilemap[(x + 1) + y * s_map->layers[i].width];
uv = get_atlas_uv(tile_atlas, tile.tile_index);
add_shape_vertex(layer_shape, vertex(x2, y2, uv.x1, uv.y1, rgba(255, 255, 255, 255)));
add_shape_vertex(layer_shape, vertex(x2, y1, uv.x1, uv.y1, rgba(255, 255, 255, 255)));
}
}
upload_shape(layer_shape);
add_group_shape(s_layer_groups[i], layer_shape);
free_shape(layer_shape);


At this point I may as well just use a list instead of a strip, I think.
Title: Re: minisphere 3.0.3
Post by: Flying Jester on April 06, 2016, 02:41:31 pm
Could you not just use the u[3], v[3] of the new tile on the burned vertex?
Title: Re: minisphere 3.0.3
Post by: Fat Cerberus on April 06, 2016, 02:45:10 pm
I think there was a reason I didn't do that, I don't remember what it was now.  I'll experiment some more later.
Title: Re: minisphere 3.0.3
Post by: Fat Cerberus on April 06, 2016, 11:21:05 pm

Could you not just use the u[3], v[3] of the new tile on the burned vertex?


Okay, you were right.  I'm assuming by uv[3] you were referring to the bottom-left corner of the next tile, since that's the corner I used:
Code: (c) [Select]
add_shape_vertex(layer_shape, vertex(x2, y2, uv.x1, uv.y2, rgba(255, 255, 255, 255)));


And all the rendering artifacts seem to have gone away.

Sometimes it's a bit tough to visualize how the triangle strip will go.  I'm used to thinking in terms of quads, but graphics cards only render triangles. :)
Title: Re: minisphere 3.0.3
Post by: Fat Cerberus on April 07, 2016, 02:00:24 am
Anyway, refactoring the map engine to use Galileo turned out to be a good idea.  I discovered a huge memory leak in the process that was not found due to lack of testing--when shapes were freed, their list of vertices was being leaked.  Kefka's Revenge ended up leaking over 3GB worth of vertices within seconds!

By the way, in case of animated maps, the map needs to be recalculated every time a tile changes.  You say Shapes are cheap to create.  Are they cheap enough that recalculating and uploading all the vertices for the entire map potentially every frame won't be an issue?  I assume that the vastly lower number of draw calls required will more than make up the difference, but I want to make sure.
Title: Re: minisphere 3.0.3
Post by: Flying Jester on April 07, 2016, 03:41:44 am
If Allegro is implementing its primitives similar to Sapphire, then it's as cheap if not cheaper to recreate Shapes than it is to modify texture coordinates.

In OpenGL, it's the same cost or more expensive to modify a buffer than as it is to upload the same data to a new buffer.

I would, however, expect that it's cheaper to play with texturing (possibly in shader, maybe swapping textures) than to create a new map of Shapes.
Title: Re: minisphere 3.0.3
Post by: Fat Cerberus on April 07, 2016, 10:27:48 am
I believe that al_create_vertex_buffer() is just a thin wrapper over OpenGL VAOs, so yeah, it should be similar in performance characteristics to Sapphire.  Although for older OpenGL versions where that function will fail I fall back to storing the vertices in a malloc'd buffer.  In that case I'm uploading more vertices per draw than will be visible, but I assume I'm unlikely to encounter such hardware that often these days, so I won't worry about it.

I do have to refactor a few things now, though, to maintain perfect emulation.  SetTileImage() is currently implemented by swapping out the image for a tile with the specified one.  Normally a tileset is a list of subimages of the tile atlas, any one of which can easily be replaced by an independent image (by the way: making an image which is a slice of another image is an awesome Allegro feature and greatly simplifies atlas management).  But that won't work now--instead the function will need to modify the atlas directly.

I guess the next thing I should do is refactor text rendering.  That's currently one of minisphere's weak points as discovered by Radnen way back in the early versions.  Rendering whole strings as a single primitive should improve it greatly.  It might even fix the performance issues I'm seeing in Specs' battle engine!
Title: Re: minisphere 3.0.3
Post by: Flying Jester on April 07, 2016, 11:02:53 am

I believe that al_create_vertex_buffer() is just a thin wrapper over OpenGL VAOs, so yeah, it should be similar in performance characteristics to Sapphire.  Although for older OpenGL versions where that function will fail I fall back to storing the vertices in a malloc'd buffer.  In that case I'm uploading more vertices per draw than will be visible, but I assume I'm unlikely to encounter such hardware that often these days, so I won't worry about it.


That would be really old OpenGL. I would be quite surprised if any machines that can even work with Allegro didn't have VBOs (which are still used as the underlying storage API for VAOs, and have similar performance characteristics in this regard).
Title: Re: minisphere 3.0.3
Post by: Radnen on April 07, 2016, 01:12:37 pm

I guess the next thing I should do is refactor text rendering.  That's currently one of minisphere's weak points as discovered by Radnen way back in the early versions.  Rendering whole strings as a single primitive should improve it greatly.  It might even fix the performance issues I'm seeing in Specs' battle engine!


This is an easy way to get huge performance benefits. :) Put each character into an atlas and then construct a single vertex map of the text to draw on screen. It's like a mini tiling engine, but with font characters rather than map tiles.
Title: Re: minisphere 3.0.3
Post by: Harry Bo21 on April 08, 2016, 01:30:57 pm
Hey all

been a while since ive been about here, saw all the improvements and was quickly tempted back

so thought id try minisphere

(https://i.gyazo.com/aea644db7eb2199d03d9822d0fa29720.png)(https://i.gyazo.com/3d0c01543984316bbc8182cc03f963f3.png)

and of course immediately hit issues? what were once common functions are unrecognized?

Is there a good tutorial for me to go revisit, like i said its been a while. This project works just fine on 1.5 yet everything is "undefined" when i try via minisphere? Even functions i made within this same script its saying are undefined?


Ive hunted around but honestly i cant find any info on getting started again
Title: Re: minisphere 3.0.3
Post by: Fat Cerberus on April 08, 2016, 01:33:22 pm
That should be CreatePerson. (capital C).  Also based on that screenshot, you're running Sphere 1.5, not minisphere? ???
Title: Re: minisphere 3.0.3
Post by: Harry Bo21 on April 08, 2016, 01:35:05 pm
well i tried the other first, forgot this was case sensitive lol

im guessing this is not the right place for me to be asking also right? Should i be in the support section? Ive been away for years
Title: Re: minisphere 3.0.3
Post by: DaVince on April 08, 2016, 01:56:21 pm

well i tried the other first, forgot this was case sensitive lol

im guessing this is not the right place for me to be asking also right? Should i be in the support section? Ive been away for years

Yes, but also just keep the function reference (http://wiki.spheredev.org/API:Functions) open at all times to aid you. :)
Title: Re: minisphere 3.0.4
Post by: Fat Cerberus on April 09, 2016, 01:28:07 am
@DaVince: Currently waiting on Launchpad to publish 3.0.4 for Trusty, after which I'll copy it to the later distros.  I found out that Ubuntu will auto-update PPAs, which is very convenient.  Once 3.0.4 is available, could you test it out for me?  I'm curious to see if the Galileo map rendering is an improvement in practice.  It's a bit hard for me to get meaningful benchmarks on this thing, it's a Core i7-6700HQ laptop with a 4GB nVidia GTX graphics card, there's not a lot minisphere can throw at it that it won't take like a champ. :P  (edit: 3.0.4 is published)

@Eggbert: Could you update the AUR package?  It's still at 3.0.1 and I've fixed a few important bugs since then.
Title: Re: minisphere 3.0.4
Post by: DaVince on April 09, 2016, 09:43:49 am
Tested!
My test case was as follows:
I spawned 1500 (fully functional, animating) cupcakes in Sir Boingers.

On 3.0.2: about 25 FPS.
On 3.0.4: fluctuates around 21-24 FPS. Hmmm.
(And bonus: regular Sphere 1.5 flat out crashes after an insane amount of lag. Damn.)

I only realized your rendering techniques are good for layer rendering and not sprite counts afterwards, but I'm not really sure how to stress test that and it's a bit late now since I'm on 3.0.4 now. :P I can still get 3.0.2 or whatever from Github and test a huge map with many animated tiles or whatever, if you like.
Title: Re: minisphere 3.0.4
Post by: Fat Cerberus on April 09, 2016, 09:48:08 am
Yeah, sprites are still drawn individually.  However Allegro /should/ be batching those if they come from the same spriteset, so I'm not sure what's causing the slowdown...
Title: Re: minisphere 3.0.4
Post by: DaVince on April 09, 2016, 09:51:52 am
The slowdown is probably because there are 1500 sprites pushing COMMAND_ANIMATE to the command buffer all at once. The 1-4 frame difference *might* be the new rendering, but who truly knows?

I'll try making a huuuuuge map with many layers and randomly animated tiles to see if I can stress test the actual map rendering in any way.
Title: Re: minisphere 3.0.4
Post by: DaVince on April 09, 2016, 10:04:27 am
Okay, something's obviously not right. All I did was resize a map to 500x500 size, and these are the results:

On 3.0.2, it runs smoothly at 60 FPS.
On 3.0.4, it drops way down to 32 FPS.
Title: Re: minisphere 3.0.4
Post by: Fat Cerberus on April 09, 2016, 10:06:56 am
Ouch!  Is that true even if the map has no animated tiles?
Title: Re: minisphere 3.0.4
Post by: DaVince on April 09, 2016, 10:18:21 am
Yeah, the tiles are completely static (regular Sir Boingers tileset), so I'm not sure what's going on here.
Title: Re: minisphere 3.0.4
Post by: Fat Cerberus on April 09, 2016, 10:20:58 am
Okay, could you do me a favor: Run the test with spherun --verbose 4, redirect the output to a file (it'll print a lot of stuff!), and look for either "uploading shape" or "caching vertices" in the log.  If you see the latter, then I think I know what's wrong.
Title: Re: minisphere 3.0.4
Post by: Fat Cerberus on April 09, 2016, 11:52:35 am
@DaVince: I figured out the problem.  The Ubuntu build uses Allegro 5.0, which doesn't support vertex buffers.  This means all the vertices for the entire map have to be uploaded to the GPU every frame.  Which of course is slower than before because originally only vertices for visible tiles had to be sent.  If you have a 500x500 map, that comes out to over a million vertices per frame!  I'll have to add code to fall back on the old method when building against Allegro 5.0.

Alternatively, I might be able to call into OpenGL APIs directly, but that would need some refactoring of the image handling, so probably not worth the effort.

Allegro 5.2 is out, so it's no longer considered "unstable", but the ball is still in Debian's court to update their Allegro packages.  In the meantime, I still have to support 5.0. :-\
Title: Re: minisphere 3.0.4
Post by: DaVince on April 09, 2016, 12:04:27 pm
Okay, sorry I wasn't around to do that test for you, glad you figured it out though! I think I'll be compiling my own versions from Git from now on, because seriously. I want that Allegro 5.2 support. :P
Title: Re: minisphere 3.0.4
Post by: Fat Cerberus on April 09, 2016, 12:11:17 pm
Actually could you do that?  Build 3.0.4 against the Allegro PPA and do that huge map test again.  Also run the map engine with framerate = 0 so we can get a good benchmark. :)
Title: Re: minisphere 3.0.4
Post by: DaVince on April 09, 2016, 12:19:52 pm
Oh, I didn't know I could make the framerate unlimited in the map engine too. On it!

On self-compiled 3.0.2: approximately 205 FPS.
On self-compiled 3.0.4: still ~16 FPS.

I'm using the Allegro unstable PPA, which still seems to be at Allegro 5.1.13, compiled in January. I guess this is too old? Should I be getting Allegro the old-fashioned way (i.e. messing with the source tarball)? Could you maybe include an Allegro 5.2 in your PPA, or is that too much hassle? :(
Title: Re: minisphere 3.0.4
Post by: Fat Cerberus on April 09, 2016, 12:23:32 pm
No, 5.1.13 should have the vertex buffers.

I can probably statically link against Allegro.  However I suspect in this case it might be that your graphics card doesn't support VBOs and I'll have to add a fallback.  Is this the P4 machine with OpenGL 1.3?  If so that's probably the issue.
Title: Re: minisphere 3.0.4
Post by: DaVince on April 09, 2016, 12:29:42 pm
No, the P4 would never ever support that or get any high framerates. :P I'm back in possession of my ~4 year old laptop (that nonetheless still serves me fine), which uses an NVIDIA GT540m. From what I can tell, that... should support VBOs? I'm not sure how new of a feature that is... GPU definitely supports OpenGL up to 4.5 though.
Title: Re: minisphere 3.0.4
Post by: Fat Cerberus on April 09, 2016, 12:34:19 pm
Yeah, that should definitely be supported.  Minimum should be GL 3.0, the 450M says it supports OpenGL 4.5.  If this is occurring on modern hardware, I can probably reproduce it locally on my laptop's GTX 950M.  Thanks for testing, looks like 3.0.5 is in the offing. :P
Title: Re: minisphere 3.0.4
Post by: Fat Cerberus on April 09, 2016, 03:14:32 pm
@DaVince
So I did some experiments.  On my laptop, I can create the VBOs successfully, and then even the 500x500 map is rendered fast (= literally no difference in CPU usage between that and 50x50).  If I artificially disabled vertex buffer support, the framerate drops to 10-20fps.  So what it looks like is, for whatever reason minisphere is failing to create the vertex buffer for you, and falling back on storing the vertices in system memory.  So now you're sending 1M+ vertices to the graphics card every frame (for the 500x500 map), which is very slow.

That said, turning vertex buffer support back on still only got me to 40fps for a 500x500 map in Sir Boingers.  In that case the animated tiles were to blame--every time a tile changes the map needs to be recalculated.  But I think I have a clever way to get around that.

Sorry about the performance regression--this is actually kind of embarrassing. :-[
Title: Re: minisphere 3.0.4
Post by: Radnen on April 09, 2016, 04:34:24 pm
DaVince, can you test my Sphere-SFML? I know I'm not really developing it right now, but I'm just curious of the performance, seeing if a faster script engine is better when it comes to things like command animate, or if that isn't the bottleneck.
Title: Re: minisphere 3.0.4
Post by: DaVince on April 09, 2016, 06:14:58 pm
Lord English: not sure what's going on on my side, then... By all means it SHOULD make the VBOs. :(

Radnen: I'm not exactly in a position to do that right now, since my Windows install is pretty broken! I'll try once I fix that, though.
Title: Re: minisphere 3.0.4
Post by: Flying Jester on April 09, 2016, 06:30:02 pm

That said, turning vertex buffer support back on still only got me to 40fps for a 500x500 map in Sir Boingers.  In that case the animated tiles were to blame--every time a tile changes the map needs to be recalculated.  But I think I have a clever way to get around that.

Sorry about the performance regression--this is actually kind of embarrassing. :-[


My solution to this was to just make shapes for every combination of animations. It avoids the main issue, but I do think there must be a better solution to this.

It also means that once animated tiles start their animations at a non-zero time (such as when SetTile is called) you start to get very large numbers of shapes.

An alternative would be to simply make each animated tile its own shape, but I never really explored that. Perhaps a combination of this and per-animation-frame shapes would work well.

EDIT: I might add that it should not be an unreasonable requirement to use VBOs. This sounds like an issue in Allegro.
Title: Re: minisphere 3.0.4
Post by: Fat Cerberus on April 09, 2016, 06:47:47 pm
My idea was quite different: I have the tileset loader create two images, one for the original atlas, and another which is used as the texture for actual rendering.  When the tileset is loaded, the texture is a perfect copy of the atlas.  When a tile animates I use render-to-texture to update the texture image by blitting the proper tile from the atlas.  This way I can calculate the map vertices once, at map load, without sacrificing animated tile support.  SetTileImage() would of course do the same thing, updating the texture instead of recalculating the map.

SetTile() would still have to recalculate the map, but that shouldn't be a huge issue as a game hopefully won't be calling that every frame!

This approach of course uses more VRAM, but since you can only have one tileset loaded at a time, it shouldn't be an issue.  And for non-animated tilesets I can have a copy-on-write scheme to avoid cloning the atlas unnecessarily.
Title: Re: minisphere 3.0.4
Post by: Fat Cerberus on April 09, 2016, 07:30:07 pm
My render-to-texture method worked, and improved performance with animated tiles a ton. :)  Now I just have to figure out why it's failing to create the vertex buffer...
Title: Re: minisphere 3.0.4
Post by: Radnen on April 09, 2016, 08:30:35 pm

My idea was quite different: I have the tileset loader create two images, one for the original atlas, and another which is used as the texture for actual rendering.  When the tileset is loaded, the texture is a perfect copy of the atlas.  When a tile animates I use render-to-texture to update the texture image by blitting the proper tile from the atlas.


I do this exact method in SSFML! :D I blit from the original texture map straight to a copy that is the texture to render to screen. The copy was needed since you're always overwriting animated tiles and lose information. I found it helped out by a solid margin.
Title: Re: minisphere 3.0.4
Post by: Fat Cerberus on April 10, 2016, 01:09:36 am
Hm, I'm starting to question whether the VBO creation is actually failing.  With my laptop set to use the i7's iGPU, if I resize the Boingers tutorial map to 500x500, I can hit around 100 fps.  At its normal size of 100x15, that goes up to 1600+ fps (!).  It looks like if you give the GPU a million vertices, it gets bogged down regardless of whether they're cached in a VBO or not--even though they are not all visible on the screen.  If I had to guess, I would say it's vertex shader overhead and if that's the case, there's not much to be done for it other than avoiding creating huge 500x500 maps. ;)

I guess this is one case where map stitching would come in handy.  You could stitch smaller maps together to create one huge one and just avoid drawing the ones which are off-screen.

edit: Actually... that gives me an idea!  Rather than creating one large triangle strip for an entire map, I could slice it up into screen-sized rectangles and only draw the slices which are actually visible.  If we assume a 320x240 screen with 16x16 tiles (or equivalently, 640x480 with 32x32 tiles), each slice would be 20x15 tiles, or ~1500 vertices.  In the worst case we have to draw 4 of those, so that's 6000 vertices--which remains constant regardless of the overall size of the map and is quite a bit less than a million! :P
Title: Re: minisphere 3.0.5
Post by: Fat Cerberus on April 10, 2016, 02:59:19 am
I think I'll hold off on implementing that slicing idea until the next release (3.1).  With the changes I've made in the last 24 hours, the map rendering performance should now be acceptable in almost all real-world cases.  The primary bottleneck for sanely-sized maps in 3.0.4 was animated tiles, which I've solved in 3.0.5.
Title: Re: minisphere 3.0.4
Post by: Radnen on April 10, 2016, 04:28:39 am

edit: Actually... that gives me an idea!  Rather than creating one large triangle strip for an entire map, I could slice it up into screen-sized rectangles and only draw the slices which are actually visible.  If we assume a 320x240 screen with 16x16 tiles (or equivalently, 640x480 with 32x32 tiles), each slice would be 20x15 tiles, or ~1500 vertices.  In the worst case we have to draw 4 of those, so that's 6000 vertices--which remains constant regardless of the overall size of the map and is quite a bit less than a million! :P


I do that too! But for the Sphere editor, and instead of screen sized slices, the slices are much smaller. But think of Google Maps. There are thousands of small panels that stream in as you pan the mouse.

The only other thing I could do for performance is add asynchronous panel loading (adding multi-threading to it).
Title: Re: minisphere 3.0.5
Post by: Fat Cerberus on April 10, 2016, 11:11:46 am

I do that too! But for the Sphere editor, and instead of screen sized slices, the slices are much smaller. But think of Google Maps. There are thousands of small panels that stream in as you pan the mouse.

The only other thing I could do for performance is add asynchronous panel loading (adding multi-threading to it).


For realtime rendering, it's a bit of a tradeoff.  Prior to 3.0.4, minisphere rendered maps as individual quads, one for each tile, and only for those tiles which are actually visible.  In some ways, this is good: You get tile animation basically for free just by modifying the internal image index, you don't even have to blit images around.  But it's not really optimal; if your visible slice is 20x15 tiles, that comes out to ~300*num_layers draw operations per frame.  By using screen sized slices, you only have to issue at most 4 draws per layer.  In any case, as I found out, you don't want to be drawing a whole map's worth of vertices if that map is larger than about 200x200, otherwise you start seeing performance degradation.  Worse, the loss of performance is proportional to the size of the map, where ideally you want it to be constant--or at least proportional to the screen resolution instead.

At first the performance regression for large maps didn't make sense to me.  I figured, okay, the vertices are already in VRAM and if a triangle is not visible, the GPU can just skip it.  But I realize now why it happens: The GPU can't actually guarantee the triangle is not visible until it pushes all of the vertices through the vertex shader first, which may relocate them.  You can't just throw 250,000 quads worth of vertices at the pipeline and not expect it to get bogged down, especially on less capable hardware. :P
Title: Re: minisphere 3.0.5
Post by: N E O on April 10, 2016, 12:05:55 pm

I guess this is one case where map stitching would come in handy.  You could stitch smaller maps together to create one huge one and just avoid drawing the ones which are off-screen.


Wait, you weren't ignoring rendering off-screen tiles this whole time???
Title: Re: minisphere 3.0.5
Post by: Fat Cerberus on April 10, 2016, 12:10:34 pm


I guess this is one case where map stitching would come in handy.  You could stitch smaller maps together to create one huge one and just avoid drawing the ones which are off-screen.


Wait, you weren't ignoring rendering off-screen tiles this whole time???


I did ignore them prior to 3.0.4, but the map was drawn tile-by-tile.  The switch to Galileo-based rendering in 3.0.4 changed it so that I calculate a mesh for the entire a map ahead of time, and then issue a single draw operation.  For maps up to around 200x200, this is fine even on ~5-year-old hardware.  Any more than that starts to bog down the graphics pipeline with vertices.
Title: Re: minisphere 3.0.5
Post by: Fat Cerberus on April 10, 2016, 07:15:25 pm
So, funny story: Allegro is smarter than I gave it credit for.  It has a sprite batcher built in which is actually awesome and it turns out that didn't need to change the map rendering at all--in fact, I ended up hurting performance by changing it.  See, it turns out that if you do this:

Code: (c) [Select]

al_hold_bitmap_drawing(true);
// draw all visible tiles using al_draw_bitmap()
al_hold_bitmap_drawing(false);


As long as all specified images come from the same underlying texture atlas, Allegro will coalesce all the quads into one triangle list and perform a single draw operation to render them.  You basically get Galileo-level rendering performance for free.
Title: Re: minisphere 3.0.5
Post by: Radnen on April 10, 2016, 09:57:15 pm
I thought you were already doing that? :/
Title: Re: minisphere 3.0.5
Post by: Fat Cerberus on April 10, 2016, 10:02:20 pm
I was--before 3.0.4.  I misguidedly thought using the Galileo API would improve performance.  Needless to say it didn't. :-\
Title: Re: minisphere 3.0.6
Post by: Fat Cerberus on April 11, 2016, 12:40:24 am
To clarify a bit on the above, I already knew beforehand that al_hold_bitmap_drawing() did some sprite batching and took advantage of it.  However I assumed it was just basic grouping -- i.e. to avoid rebinding textures when drawing multiple images from an atlas.  In reality, it turns out it's a bit more intelligent than that: It automatically does exactly what I was trying to accomplish with the Galileo routines (i.e. collapsing everything into a single draw operation), and does it better than I was able to do manually.  And this is why minisphere 3.0.6 returns to using that method. :)
Title: Re: minisphere 3.0.6
Post by: Fat Cerberus on April 12, 2016, 02:44:57 am
So, I fired up the profiler and figured out the cause of minisphere's slow text rendering: While I do keep the RFN glyphs in an atlas and batch all the characters, the primary bottleneck seems to be the cost of populating a VBO every frame.  By looking at the Allegro source, I found out that, when using al_hold_bitmap_drawing() and drawing from the same texture, it builds up a primitive containing quads for every image drawn and once the hold is released, uploads that into a VBO and draws it.  Since this is happening every frame, it gets pretty expensive.

I'm not sure there's much to be done for this on the engine side.  It's just a fact of how hardware graphics acceleration works.  For game code there are ways around it of course.  One solution would be to create a Surface, render the desired text into it, and just draw that every frame.  Another, less memory-hungry way would be to use the Galileo API, create a Shape for the desired text and cache it.  This way the VBO only needs to be created once.
Title: Re: minisphere 3.0.8
Post by: Fat Cerberus on April 17, 2016, 02:47:05 am
minisphere 3.0.8 is up.  This is a quick update to add support in SSJ for repeatable commands.  This is a feature inspired by gdb - after typing, say, "stepover", you can keep pressing Enter to step through the code line-by-line, instead of having to type "s" each time.

The commands that can be repeated this way are:


I also made a slight usability enhancement related to this: If the SSJ command-line is left blank with no command to repeat, it will give the user a hint to type "help".  This should help out newbies a bit in learning the debugger commands.
Title: Re: minisphere 3.0.8
Post by: DaVince on April 17, 2016, 11:43:55 am
There haven't been many replies lately, but I'd just like to say good work on all the progress and improvement. :)
Title: Re: minisphere 3.0.8
Post by: Fat Cerberus on April 17, 2016, 11:45:58 am
By the way, @DaVince, did you stress-test 3.0.7 at any point?  Well I guess now it would be 3.0.8... :P
Title: Re: minisphere 3.0.8
Post by: DaVince on April 17, 2016, 12:05:47 pm

By the way, @DaVince, did you stress-test 3.0.7 at any point?  Well I guess now it would be 3.0.8... :P

It'd forgotten about it until you told me just now, but spawning a cupcake every 3 pixels on the X axis 1500 times slowed things down to 14-21 FPS before, while it stays at a smooth 60 FPS now. Wonderful. :)
Title: Re: minisphere 3.0.8
Post by: Fat Cerberus on April 28, 2016, 08:07:15 pm
I made some enhancements to the Galileo API to make it easier to do transformations and such.  Instead of X/Y/angle properties, I added a group.transform property which is used to work with the transformation matrix directly.  This allows you to do more complex transformations:

Code: (javascript) [Select]

group.transform.identity();
group.transform.scale(2, 2);  // 2x scale
group.transform.translate(10, 0);  // x += 10
group.transform.rotate(Math.PI * 2 / 4);  // revolve 90 degrees
group.draw();


Other enhancements I made in the last commit: Drawing Shapes directly (without a group), and the ability to do Galileo rendering, both Shapes and Groups, onto a Surface instead of the backbuffer.  So those enhancements will be in minisphere 3.1, whenever I decide to release that.

Finally, I have an experimental branch on GitHub (pegasus-api) that overhauls the entire API to be more Pegasus-like and removes a lot of legacy baggage from the core engine such as the classic map engine, ByteArrays, etc.  I'm still unsure if I want to release the Pegasus modifications with minisphere 4.0, or fork it off as a seperate "Sphere 2.0" repo.  Either way, I might release an alpha version of that soon, so keep an eye out for it.  In the meantime minisphere will remain backwards compatible for the foreseeable future. :)
Title: Re: minisphere 3.0.8
Post by: Fat Cerberus on May 06, 2016, 08:31:17 am
I took a long look at Cell yesterday and I have to concede that the experiment was a failure.  It tends to just add an inconvenient build step to the Sphere workflow for no real gain.  It's easy enough to write a .s2gm file by hand (it's just JSON) and Sphere's strength, like most JS runtimes, has always been the ability to run code directly with no formal preparation needed.  For more complex workflows, it'd probably be better to have small, purpose-built tools for each task (such as tileset/spriteset generation, etc.) so they can be assembled into a batch file or shell script as needed, leaving the "all-in-one" type stuff to IDEs like Sphere Studio :)

Anyway I'm going to repurpose the program.  Instead of being a scriptable compiler (a task it does very poorly now), it will be remade into a project management tool and SPK packager.  To start a new Sphere project you run cell init in an empty directory, which creates the .s2gm file.  From there you can do cell add/remove filename to add and remove files from the project (this is needed so Cell knows what to include in the distribution), and cell package to compress it all into an SPK.  There will be more commands available too (such as to set the title, author, API level, etc.), but that's the basic workflow. This will allow Sphere games to be developed entirely on the command line if so desired.
Title: Re: minisphere 3.0.8
Post by: DaVince on May 06, 2016, 03:07:17 pm
That sounds really useful, especially on Linux. Think you can also generate the standard project folders on cell init?

Also, for the "adding files" thing, I don't know how I feel about manually adding every thing into the project. It adds a factor of forgetting to add that new script or resource you just made that the game uses. Wouldn't adding every file/folder found that matches the standard Sphere format work better? Or perhaps a list of files to ignore for packaging, rather than files to add?
Title: Re: minisphere 3.0.8
Post by: Fat Cerberus on May 06, 2016, 03:26:39 pm

That sounds really useful, especially on Linux. Think you can also generate the standard project folders on cell init?

Also, for the "adding files" thing, I don't know how I feel about manually adding every thing into the project. It adds a factor of forgetting to add that new script or resource you just made that the game uses. Wouldn't adding every file/folder found that matches the standard Sphere format work better? Or perhaps a list of files to ignore for packaging, rather than files to add?


Hm, I agree it's very easy to forget to add something, and then end up with a non-working SPK package.  Automatically adding all files matching the Sphere formats isn't really optimal either, since there are some games (like your own Sir Boingers) that have .txt files serving as control data in the maps folder that would be skipped over and break the package.  So the blacklist idea might work.  To simplify things it could be based on wildcard patterns as with the .gitignore file in a git repository rather than individual filenames.

The other alternative is to just package everything in the source directory, but that requires some discipline on the developer's part to put non-game files like design documents and such outside of the source directory.

I do like the idea of having it create the standard folders.  Thanks for the tip. :D
Title: Re: minisphere 3.1.0
Post by: Fat Cerberus on May 07, 2016, 03:08:19 am
minisphere 3.1.0 is up.  I'm not up for that Cell overhaul just yet, but didn't want to delay the 3.1.0 release, so... here it is!  New in this version: the Transform object, setting shader uniforms (Allegro 5.2 only), new single-character SphereFS aliases, new miniRT modules, and a few other enhancements under the hood.  Get it while it's hot! :D
Title: Re: minisphere 3.1.0
Post by: Fat Cerberus on May 07, 2016, 10:10:24 am
From here on out I'm going to try to minimize the breaking changes in minor releases.  It's going to make Sphere very hard to support from a user perspective if games break with every update.  It's not so bad for us developers who can just update our code, but breaking packaged games is not nice.  The only thing that's been perfectly stable since the first version of minisphere is the Sphere 1.x API for the simple fact that it HAS to be locked down to avoid breaking compatibility.

This is part of why I want to implement an Android-like "API level" value in the eventual overhaul.  The level is strictly increasing so that whenever changes are made to the API, the API level is bumped.  This makes writing compatibility shims easier as well as allowing game code to adapt more easily since each API level is well-defined, as opposed to the current model of, "Oh, I think I changed that in minisphere 8.12... or maybe it was 7.0?  Anyway, yeah, go look at the API reference again." :P

Anyway, I'll try to have more self-control going forward.
Title: Re: minisphere 3.1.1
Post by: Fat Cerberus on May 10, 2016, 03:15:44 am
@DaVince minisphere 3.1.1 includes a change that might fix the lockup you saw the other day when running against Allegro 5.0.  Care to re-test when you get a chance?
Title: Re: minisphere 3.1.1
Post by: Rahkiin on May 10, 2016, 10:34:53 am
You could just document when you add or deprecate a function/value/class/whatever.

I suggest you use SemVer: breaking changes (removing stuff) needs a major version number update, new stuff needs a minor version update (old code still works, but code written for 5.4 might not work on 5.3). And when you fix stuff, use patch. 5.4 code should work on 5.4.10 and other way around.
Title: Re: minisphere 3.1.1
Post by: Fat Cerberus on May 10, 2016, 11:47:18 am
Yeah, the problem is that you basically can't ever break anything once the API is established, otherwise you break already-packaged and released games which isn't nice.  It'd be like if every single Android update broke everything on the app store.  People wouldn't think very highly of the platform. :P  As for SemVer, it's actually really hard to get right.  I refactor constantly, and since I'm a perfectionist the refactoring usually extends to the API, so stuff breaks.  If I followed SemVer to the letter, you'd be getting minisphere v812.0 by now. :P
Title: Re: minisphere 3.1.1
Post by: Rahkiin on May 10, 2016, 03:37:12 pm
Just don't release so often then. Also, if you give someone a game, pack it with the version of minisphere you tested it with.
If you send someone a game with my engine, the .app contains both the game and the engine so you can just start it. And it is always the right version, until the developers gives the game an update with new version of the engine and distributes that.
Title: Re: minisphere 3.1.1
Post by: DaVince on May 10, 2016, 03:48:03 pm

@DaVince minisphere 3.1.1 includes a change that might fix the lockup you saw the other day when running against Allegro 5.0.  Care to re-test when you get a chance?

Works great, thanks. :)
Title: Re: minisphere 3.1.1
Post by: Fat Cerberus on May 10, 2016, 03:48:50 pm
Nah, I follow the philosophy of Release Early, Release Often, I don't like keeping new features to myself and this is more likely to reveal bugs quickly :)

Anyway, packaging the engine was something normally done with Sphere 1.x.  I don't much like that - it'd be nicer if users could just install one copy of the engine and then you only have to distribute the .spk file.  This can be done as long as I show some self control in changing public APIs.  Which is exactly what I promised to do above, so don't worry.
Title: Re: minisphere 3.1.1
Post by: Fat Cerberus on May 10, 2016, 03:50:47 pm


@DaVince minisphere 3.1.1 includes a change that might fix the lockup you saw the other day when running against Allegro 5.0.  Care to re-test when you get a chance?

Works great, thanks. :)


That's good.  I wish I knew the workaround would be so simple or I would have done it a long time ago. ;)

By the way you can do "spherun --version" to see the version of all dependencies including Allegro.  Useful when making bug reports since everything on Linux is dynamically linked.
Title: Re: minisphere 3.1.2
Post by: Fat Cerberus on May 15, 2016, 03:17:24 am
So I decided what I want to do for minisphere 4.0.  I'm going in a slightly different direction than the original Pegasus ideals.  In Pegasus, everything is a module, including all the standard subsystems.  This is similar in design to Node.js.  Now in Node this kind of setup makes sense, because it's a primarily designed for servers and you only want your server to load what it actually uses.  Node has a LOT of different components built into it that most apps won't even use half of, so it would be wasteful to load up everything in advance.

In all my experience with Sphere, I can't help but feel that the Node.js-inspired design for Pegasus is overkill.  It's not a big deal if the engine needs to initialize audio for a game that happens not to play sound, for example.  And needing to, say, require("graphics"); at the top of every file gets tedious, even more so once you start needing other components (audio, engine, fs, keyboard...).  In practice all that accomplishes is make you want to write a do-it-all wrapper module to avoid all the boilerplate every time you create a new file.  Which entirely defeats the purpose of modularizing it.  :P

This doesn't mean I'm doing away with modules, mind you.  Being able to write a self-contained script that's guaranteed not to clash with any other library regardless of what you name your exports?  Pure unadulterated awesome.  So instead, my idea is for low-level stuff to be exposed globally, as in Sphere 1.x, while the more high-level "scripty" stuff gets put into modules.  For example, you can require("prim") to get convenient Sphere 1.x-like immediate-mode primitives:
https://github.com/fatcerberus/minisphere/blob/sphere-api-v2/assets/system/modules/prim.js

Or require("link") for Radnen's awesome query library. :D  For Scenario, you'd require "scenes", etc.  That kind of code makes sense as a module, but the core engine functionality is better if it's exposed by default.  That's, I think, a big reason for Sphere's high level of accessibility over the years, and I wouldn't want to give that up.
Title: Re: minisphere 3.1.2
Post by: Rahkiin on May 15, 2016, 09:01:10 am
So what are you going to do? You don't actually mention that.

My engine just always loads the audio and graphics stuff. You just load an API using the require() stuff... it is not that only then the graphics is set up.

I think it is nice to have modules. You could then easily extend api's or override them as a user. So instead of using 'graphics' y ou could do 'let graphics = require('./myGraphics')' which loads an API that has the same API but does stuff differently. It allows for modulism.

I would like to hear what else you propose. If you throw it all in global space, you can't override them anymore: not using modules anyways. Always only within the same module...
Title: Re: minisphere 3.1.2
Post by: Fat Cerberus on May 15, 2016, 10:26:56 am
I thought I explained it plenty well: Low-level functionality would be global, anything "user replaceable" is a module.

The thing is, you probably aren't going to override the native functions because they're so low-level.  There are no "user-replaceable parts" there.  It's impossible to replicate the basics of Galileo in script, for example.  So if you're not changing the API, what could a replacement graphics module actually "do differently"?  All the higher level stuff that you would want to swap out like the map engine, cutscene manager, etc. is still going to be in modules.

That's why I'm designing the core to be so low-level by the way - the engine provides the bare minimum functionality to make a useful game out of the box, and all modules are built on top of that.

Think of how annoying it would be if, in JS, you had to require("array") in every file that worked with arrays.  Or require("string") for strings, etc.  It would be infuriating.  That's the level of functionality I'm talking about making global, the basic building blocks.  Everything else is modular.
Title: Re: minisphere 3.1.2
Post by: Rahkiin on May 15, 2016, 10:29:47 am
Hmmm.

So you will be adding a large bunch of globals? Make sure they are all classes and start with capitals.

Could you give me the classes you will expose?
Title: Re: minisphere 3.1.2
Post by: Fat Cerberus on May 15, 2016, 11:47:13 am

So you will be adding a large bunch of globals? Make sure they are all classes and start with capitals.


Not nearly as many as you're probably imagining. ;)  It won't be anything like the Sphere 1.x API, that's for sure.

For reference, here's a list of every single global in minisphere 3.1.2:
https://github.com/Radnen/spherestudio/blob/master/ScriptEditPlugin/Dictionary/SphereAPI.txt

minisphere 4.0 won't even have a fraction of that.  Like I said, I still very much prefer modules for high-level functionality.  Here's my current module list:
https://github.com/fatcerberus/minisphere/tree/sphere-api-v2/assets/system/modules

Global classes/namespaces are currently:
Color
Group
Image
Logger (<-- is now a module :) )
Mixer
Server
Shader
Shape
Socket
Sound
SoundStream
Surface
Transform

global singletons:
console
engine
fs
keyboard
manifest
random (<-- this can probably be a module too)

"manifest" was going to be called "game" but I found that it tended to get shadowed in practice, causing bugs.
Title: Re: minisphere 3.1.2
Post by: Flying Jester on May 15, 2016, 02:27:55 pm
Or you could do what Turbo does and make everything a plugin/module, but only require a game to specify which modules it uses in its manifest/sgm file.
Title: Re: minisphere 3.1.2
Post by: Fat Cerberus on May 15, 2016, 02:35:57 pm

Or you could do what Turbo does and make everything a plugin/module, but only require a game to specify which modules it uses in its manifest/sgm file.


Yeah, I figured I would do something like that once I implement plugins (probably a 5.0 feature).  Overall I really like the namespacing semantics of require (the caller, not the module, defines the namespace, which prevents naming conflicts entirely), but I don't want to have to add a bunch of boilerplate requires at the top of every file that needs to access functionality that's already compiled into the engine.
Title: Re: minisphere 3.1.2
Post by: Fat Cerberus on May 18, 2016, 04:32:48 pm
Progress update: I just added support for console.log() today.  Since standard output is used for engine logging, console.log() output is redirected to SSJ.  I figured it was about time I implemented this, every other JS environment known to man already includes it.

One thing I do want to do still for minisphere 3.2 is to improve the require() implementation so that I can experiment with Pegasus stuff without breaking Sphere 1.x compatibility right away.  This requires two things which are currently not supported:



At some point I may need to replace Duktape, though.  In my experiments with doing things in script, e.g. Galileo-based text rendering, minisphere's CPU usage nearly tripled compared to doing the same thing in C.  Before I do that though, it'd be nice if I could create a shim which would allow me to swap out the JS engine without having to rewrite the entire API layer.  How feasible that is, I'm not quite sure.  It may be impossible.

Duktape is a good fit for the Sphere 1.x design because the engine does all the heavy lifting--the game logic is almost never CPU bound as a result.  In a more minimal engine where even most of the infrastructure is in script, JIT is pretty much a necessity.
Title: Re: minisphere 3.1.2
Post by: Flying Jester on May 18, 2016, 06:01:48 pm

At some point I may need to replace Duktape, though.  In my experiments with doing things in script, e.g. Galileo-based text rendering, minisphere's CPU usage nearly tripled compared to doing the same thing in C.  Before I do that though, it'd be nice if I could create a shim which would allow me to swap out the JS engine without having to rewrite the entire API layer.  How feasible that is, I'm not quite sure.  It may be impossible.


The big issue I found is that different JS engines allow different specific hooks for accessing variables, modifying globals, etc.

It's certainly possible, but the amount of work it represented was literally more than changing all of TurboSphere over from V8 to SpiderMonkey. That makes the benefit of such a system, beyond very basic usage is quite questionable to me, given the amount of work to maintain it.
Title: Re: minisphere 3.1.2
Post by: Rahkiin on May 19, 2016, 02:29:51 pm

At some point I may need to replace Duktape, though.  In my experiments with doing things in script, e.g. Galileo-based text rendering, minisphere's CPU usage nearly tripled compared to doing the same thing in C.  Before I do that though, it'd be nice if I could create a shim which would allow me to swap out the JS engine without having to rewrite the entire API layer.  How feasible that is, I'm not quite sure.  It may be impossible.


That is pretty much impossible. Also, V8 and SM require C++ and works very differently regarding memory management. Don't start.

Why not just keep more stuff in C then? Why move everything to JS, if C is faster.
Title: Re: minisphere 3.1.2
Post by: Fat Cerberus on May 19, 2016, 02:39:41 pm


At some point I may need to replace Duktape, though.  In my experiments with doing things in script, e.g. Galileo-based text rendering, minisphere's CPU usage nearly tripled compared to doing the same thing in C.  Before I do that though, it'd be nice if I could create a shim which would allow me to swap out the JS engine without having to rewrite the entire API layer.  How feasible that is, I'm not quite sure.  It may be impossible.


That is pretty much impossible. Also, V8 and SM require C++ and works very differently regarding memory management. Don't start.

Why not just keep more stuff in C then? Why move everything to JS, if C is faster.


Because Duktape's stack-based API is tedious to work with when dealing with object properties.  High-level things such as the map engine would be much easier to maintain if they're written in JS.  To get an idea of what I mean, see my Node.js-like module loader:
https://github.com/fatcerberus/minisphere/blob/master/src/engine/commonjs.c#L9-L116

That would be about 20 lines of code in JS.  In C, it's nearly 100 because of all the stack juggling.  By the way, wrapping the module code in a function is exactly what Node.js does:
https://github.com/nodejs/node/blob/master/lib/internal/bootstrap_node.js#L426-L429

There's another good reason to write stuff in JS, too: If we end up with multiple engine implementations, as long as they implement the same low-level API, the same high-level JavaScript code can be reused in all of them.  This way the lion's share of the code only needs to be written once, as a "reference implementation" of sorts.
Title: Re: minisphere 3.1.2
Post by: Flying Jester on May 19, 2016, 02:48:45 pm
JS is also good enough for most things. Given how many things in the map engine are not really performance dependent (command queues, movement, scrolling) it makes sense to put them in JS.

I can also say that it just feels more natural to write the map engine in JS than in C/C++. It also makes it easy to extend the map engine.
Title: Re: minisphere 3.1.2
Post by: Rahkiin on May 21, 2016, 06:44:12 am
regarding the function() wrapper in node: that is NativeModule only. Actual modules are loaded like I do it: using a new context. This NativeModule is modules with coded already in the node executable. Totally different case.
Title: Re: minisphere 3.1.2
Post by: Fat Cerberus on May 21, 2016, 12:18:24 pm
Hm, maybe, but the end result is the same:

pig.js:
Code: (javascript) [Select]
pig = "maggie";  // "accidentally" create a global variable
console.log("the pig ate everyone - pig.js");
require("./cow");
return 812;


cow.js
Code: (javascript) [Select]
console.log("the cow ate the pig " + pig + " - cow.js");


Output
Code: [Select]

$ node pig
the pig ate everyone - pig.js
the cow ate the pig maggie - cow.js


Note two things:


Yes, I like to reverse-engineer stuff, why do you ask? :P
Title: Re: minisphere 3.2.0
Post by: Fat Cerberus on May 22, 2016, 01:47:44 pm
minisphere 3.2.0 is out, and brings with it a new Node.js-inspired module resolver and support for console.log().  There are also a few under-the-hood improvements, including a tweak to the frame skipping logic which helps keep the frame rate steady during frameskip conditions.

The new module resolver will allow you to load any script in your game directory as a module by prefixing the module ID with @/, for example @/modules/mymodule.  Relative IDs are also supported (except in global code, e.g. the startup script) and will be resolved based on the location of the calling module.  JSON is also supported so that require('@/path/to/file.json') will decode the file to a JS object rather than trying to execute it as code.
Title: Re: minisphere 3.2.0
Post by: Fat Cerberus on May 24, 2016, 10:43:22 pm
I think I'm going to drop the on-the-fly TypeScript transpiling in the next major release.  I tried an experimental change where the engine would run every script regardless of file extension through the TS transpiler (whoever decided TS should be a proper superset deserves a medal) and Spectacles took nearly a full minute to start up.  Duktape just doesn't have the performance for it.

Too bad both SpiderMonkey and V8 have C++-only APIs...

I may not be dropping Cell in minisphere 4.0 after all.  I do still want to make that command-line .s2gm editor though, so I'll need to come up with a new name for it.
Title: Re: minisphere 3.2.0
Post by: Fat Cerberus on May 25, 2016, 03:32:20 am
I did some experimenting with V8 and let me say, my first impression was not a positive one.  It's drop-dead simple to build even on Windows (after I coerced Windows Defender into not quarantining Google's build tools anyway...), but that's as far as the "simple" goes :P  The API is ridiculously complex, and is apparently such a moving target that even Google can't keep their documentation up to date--most of the code samples given won't even compile.

I understand why FJ got fed up with it now.  Too bad SpiderMonkey isn't as simple to build.

Oh, amusing tidbit: The 32-bit minisphere binary went from 2.5MB to nearly 10MB once I linked in V8. :o  I don't think it actually qualifies for the "mini" designation anymore at that point. :P
Title: Re: minisphere 3.2.0
Post by: Flying Jester on May 25, 2016, 12:04:19 pm
V8  is simple to build? Well, that's new, it was a huge hassle before.

SpiderMonkey isn't too bad, their instructions just worked for me on Windows the last time I tried a fw weeks ago.
Title: Re: minisphere 3.2.0
Post by: Fat Cerberus on May 25, 2016, 12:32:31 pm
Yeah, you need to download Google's dev tools ("Depot Tools"), put them in the PATH, set a few environment variables, and a working build is just a few commands and about a half hour away.  I guess it's improved since you used it?

I think the commands were:
Code: [Select]

fetch v8
gclient sync
cd v8
python gypfiles/gyp_v8


And then it gives you a nice MSVC solution you can build.
Title: Re: minisphere 3.2.0
Post by: Flying Jester on May 25, 2016, 12:35:48 pm
That was the supposed procedure right when I gave up on V8, but the depot tools were basically completely broken. 'gclient sync' would just puke a bunch of errors out when I ran it, and the V8/Chromium folks said I needed to set about twenty environment variables and set an identity in a config file, and that was when I had had enough of Google's development process :P

Before that, the gyp build scripts were broken or generated broken MSVC projects most of the time. Before that, the SCons setup they had actually worked quite well. That was actually what made me want to use SCons, since everything else V8 used seemed to be a huge disaster. That and autotools is not easy to use on Windows, while SCons just works on Unix and Windows (and a few other places Python works, too).

Building SM isn't so bad, assuming you get the actual SM source release instead of building it inside gecko. Then you download the MozBuild tools, run their terminal, go to the SM directory, and it's a matter of ./configure && make.
Title: Re: minisphere 3.2.0
Post by: Fat Cerberus on May 25, 2016, 12:49:55 pm
Yeah, I had no problems building it but I threw my hands up when I realized how much of a moving target the API must be for all the documentation out there to be so horribly broken.  I can understand constantly wanting to improve (being guilty of the same with minisphere), but if you're going to break things, at least try to keep some consistency in your API design.  It seems like they must rewrite the entire API layer every couple of months.  Geez.

I'm suddenly very thankful for Duktape following SemVer so religiously. :P
Title: Re: minisphere 3.2.0
Post by: Fat Cerberus on May 25, 2016, 02:20:33 pm

Building SM isn't so bad, assuming you get the actual SM source release instead of building it inside gecko. Then you download the MozBuild tools, run their terminal, go to the SM directory, and it's a matter of ./configure && make.


Hm, you're right, it really was that simple.  But the SM 38 source release is apparently broken - I ended up having to check out the entire Gecko repo and build it from there (against the esr38 branch).  Using the source release I got errors during configure about NSPR.
Title: Re: minisphere 3.2.0
Post by: Flying Jester on May 25, 2016, 04:43:36 pm
You need to either configure it to use the system nspr (default) or configure it to build its own. Fortunately, there is an effort underway to remove the NSPR dependency soon.

Also, SM is up to 45 now.
Title: Re: minisphere 3.2.0
Post by: Fat Cerberus on May 25, 2016, 05:05:36 pm
Is it?  Reading the wiki pages I got the impression that 45 was still in development/not released yet.
Title: Re: minisphere 3.2.0
Post by: Flying Jester on May 25, 2016, 07:16:38 pm
https://developer.mozilla.org/en-US/docs/Mozilla/Projects/SpiderMonkey/releases/45 (https://developer.mozilla.org/en-US/docs/Mozilla/Projects/SpiderMonkey/releases/45)

The documentation on the MDN isn't quite up to date for 45 yet. But the changes aren't too bad from 38 (mostly removing deprecated versions of functions), and in general I've really only had a couple changes to make every release. Almost always they about half the changes are related to initialization.
Title: Re: minisphere 3.2.0
Post by: Fat Cerberus on May 28, 2016, 01:34:50 am
New feature coming in the next release (3.3): Predefined colors.  The complete X11 set (http://cng.seas.rochester.edu/CNG/docs/x11color.html) will be available, and you can access them as Color.White, Color.Red, Color.DodgerBlue, etc.  Each one is a property getter which returns a fresh color object, so that writing to the returned object doesn't modify the preset color.

Another useful enhancement is Font.Default.  This is a more Sphere 2.0 way to get the default system font.  There are also analogues for shaders (ShaderProgram.Default), audio mixers (Mixer.Default) and windowstyles (WindowStyle.Default).
Title: Re: minisphere 3.2.0
Post by: Fat Cerberus on May 28, 2016, 10:54:40 am
Since the predefined colors all have alpha of 255, I'm also adding a fade method which only scales the alpha channel so that you can do:

Code: (javascript) [Select]
var translucentBlue = Color.Blue.fade(128);


Otherwise you would have to resort to the Color constructor for translucent colors.
Title: Re: minisphere 3.2.0
Post by: Radnen on May 28, 2016, 09:12:10 pm

New feature coming in the next release (3.3): Predefined colors.  The complete X11 set (http://cng.seas.rochester.edu/CNG/docs/x11color.html) will be available, and you can access them as Color.White, Color.Red, Color.DodgerBlue, etc.  Each one is a property getter which returns a fresh color object, so that writing to the returned object doesn't modify the preset color.


+1, this is awesome.
Title: Re: minisphere 3.2.0
Post by: Fat Cerberus on May 29, 2016, 01:30:18 am
Yeah, the Color API was always one of the weaknesses in Sphere.  I got spoiled working in C# where you can just say Color.Green or whatever, which is more convenient than writing out the RGB values and also makes the code more readable.

I'm also adding Color.of() to decode HTML color codes, e.g. Color.of("#ff0000") gives you red.  It also supports X11 color names, so Color.Red, etc. are actually redundant now. :P

edit: Oh, and also color_obj.name will give you the name of a color (or ARGB hexcode, if no name matches) so you can easily save it to a file.  Then you call Color.of(name) to reconstruct the color from the saved string.
Title: Re: minisphere 3.2.0
Post by: Fat Cerberus on May 31, 2016, 12:35:16 am
I switched to xoroshiro128+ (http://xoroshiro.di.unimi.it/) instead of Mersenne Twister for the random number generator.  This is the successor to the xorshift algorithm used in modern versions of both SpiderMonkey and V8 and is faster than MT as well as apparently higher quality.  The period is smaller (2^128 vs. 2^19937) but realistically there's no way you're ever going to exhaust all 340,282,366,920,938,463,463,374,607,431,768,211,456 states. :P

edit: I'm also going to add the ability to save and restore the internal state of the RNG by reading and writing RNG.state.  You can use this to prevent the player soft-resetting a game to get a different random result, if you wanted to prevent that, or even use it to implement time travel mechanics.  For xoroshiro128+, the entire internal state is representable as a 32-digit hex string, making it easy to save and restore in a JSON file.  That's a far cry from Mersenne Twister, which has 2.5KB worth of state!
Title: Re: minisphere 3.2.0
Post by: Radnen on May 31, 2016, 02:36:51 am

edit: I'm also going to add the ability to save and restore the internal state of the RNG by reading and writing RNG.state.  You can use this to prevent the player soft-resetting a game to get a different random result, if you wanted to prevent that, or even use it to implement time travel mechanics.  For xoroshiro128+, the entire internal state is representable as a 32-digit hex string, making it easy to save and restore in a JSON file.  That's a far cry from Mersenne Twister, which has 2.5KB worth of state!


Awesome! I like the ability to save the rng so that people can't save-scum a game for awesome items.
Title: Re: minisphere 3.3.0
Post by: Fat Cerberus on May 31, 2016, 01:13:14 pm
minisphere 3.3.0 is out now.  There should be no breaking changes in this version, and it includes both the predefined colors and the RNG enhancements mentioned earlier.
Title: Re: minisphere 3.3.0
Post by: Fat Cerberus on June 02, 2016, 12:43:47 am
In minisphere 3.3, besides named colors, I combined the functions of BlendColors() and BlendColorsWeighted() into a single function: Color.mix().  The function signature is:

Code: [Select]

Color.mix(color1, color2[, w1, w2]);

    Calculates a weighted average of two colors.  `w1` and `w2` are optional
    and specify the relative weights of `color1` and `color2` respectively.  If
    the weights are omitted, the mix is 50/50.


This, combined with the new named colors and Color#fade(), makes working with colors much easier:
Code: (javascript) [Select]

TurnPreview.prototype.render = function()
{
var alpha = 255 * (1.0 - this.fadeness);
var y = -16 * this.fadeness;
SetClippingRectangle(0, y, 160, 16);
Rectangle(0, y, 48, 16, Color.Black.fade(alpha * 0.75));
OutlinedRectangle(0, y, 48, 16, Color.Black.fade(alpha * 0.125));
DrawTextEx(this.font, 24, y + 2, "next:", Color.Gray.fade(alpha), 1, 'center');
Rectangle(48, y, 112, 16, Color.Black.fade(alpha * 0.75));
OutlinedRectangle(48, y, 112, 16, Color.Black.fade(alpha * 0.125));
for (var id in this.entries) {
var entry = this.entries[id];
for (var i = 0; i < entry.turnBoxes.length; ++i) {
var turnBox = entry.turnBoxes[i];
var pictureColor = entry.color.fade(alpha);
Rectangle(turnBox.x, y, 16, 16, pictureColor);
OutlinedRectangle(turnBox.x, y, 16, 16, Color.Black.fade(alpha * 0.25));
DrawTextEx(this.font, turnBox.x + 4, y + 2, entry.name[0], Color.mix(entry.color, Color.White), 1);
}
}
SetClippingRectangle(0, 0, GetScreenWidth(), GetScreenHeight());
};


The code above was littered with CreateColor() calls before and was harder to read as a result.
Title: Re: minisphere 3.3.0
Post by: Fat Cerberus on June 05, 2016, 01:36:56 am
Heads up, guys: I'm going to break Sphere 1.x compatibility in a major way in minisphere 4.0.  I don't have an idea what the full extent of that will be yet, but it's 100% certain that existing code will not run in minisphere 4.0 without major modifications.  I will probably maintain minisphere 3.3 for a while after the v4.0 release, but I figure it's time to start looking to the future of Sphere, and there are a bunch of enhancements I've wanted to make to the way the API is structured that I haven't been able to do up to now, as it would have meant breaking legacy code.

Basically, maintaining Sphere 1.x compatibility is holding back any further modernization of the API, so I think it's high time I made a clean break.  This has actually been my plan since the project started, but I've held off because... well, we're still kind of dead here, what's the rush? :P

It will be possible to write a shim of course, I'm just not sure yet whether I'm going to be the person to write it.  It's probably not worth the effort on at this point, to be honest.  minisphere 3.3 will continue to run Sphere 1.x code even if Sphere 1.x itself eventually stops working completely. ;)
Title: Re: minisphere 3.3.0
Post by: Fat Cerberus on June 09, 2016, 03:13:14 am
I'm really excited about Sphere 2.0, guys. ;D

Here's my plan: minisphere 4.0 will provide a very minimal and low-level set of functionality, for example, Galileo gives you Images, Surfaces, Colors, Groups and Shapes and that's it.  And Images and Surfaces are themselves pretty basic: Images are read-only and can only be used as textures for Shapes; and Surfaces are dumb render targets with no drawing methods of their own (but can be converted to Images if needed).  No more legacy primitives like Rectangle, GradientCircle, etc. and in general no more huge, monolithic all-in-one API.

Of course, providing only low-level functionality out of the box would make Sphere quite unappealing to newbies.  So my solution for that is to leverage the CommonJS module system to provide extra system modules (implemented in JS) along with the engine to replace some of the Sphere 1.x "all-in-one" functionality.  This is what "system scripts" were supposed to be for Sphere 1.x, but looking at the selection of scripts available there vs. what's provided by the engine already, it... doesn't really seem to have panned out. :P

For example, take my prim module:
https://github.com/fatcerberus/minisphere/blob/master/assets/system/modules/prim.js

prim provides stuff like prim.rect() or prim.blit() which replicate the Sphere 1.x graphics functions but with a more modern approach to the API:

Code: (javascript) [Select]

const prim = require('prim');
var myImage = new Image('test.png');
// note: `screen` is a Surface object!
while (true) {
prim.blit(screen, 0, 0, myImage);
prim.circle(screen, 30, 30, 20, Color.Chartreuse, Color.Black);  // gradient circle!
screen.flip();
}


Other candidates for standard system modules are stuff like Scenario, my cooperative threading engine (basically: SetUpdateScript() on a ton of steroids), Link, etc.

This naturally also makes Sphere more newbie-friendly since we can point people at these modules instead of going straight to the low-level APIs, without us needing to bloat the core engine with high-level stuff as a compromise.
Title: Re: minisphere 3.3.0
Post by: Fat Cerberus on June 12, 2016, 10:56:24 am
Moved the Sphere 2.0 changes onto a separate branch (sphere-2) and force-pushed master back to 3.3.0.  If anyone has a clone of the repo with post-3.3 commits, it might be a good idea to reset your local copy.  While I'm still going to go forward with the changes, I'm not entirely happy with breaking backward compatibility (yes, I'm indecisive, I know :P) and want to see if there's a way I can keep the old functions and still layer the new API on top.

A big part of this is that Specs relies pretty heavily on RequireScript, and the current minisphere 4.0 code removes it in favor of require().  CommonJS semantics differ quite a lot from Sphere 1.x so I would have to restructure all of my scripts, which is quite an undertaking.  I imagine Radnen is in a similar position with Blockman.
Title: Re: minisphere 3.3.0
Post by: Fat Cerberus on June 14, 2016, 03:32:58 am
Here's a sneak peak at what the Sphere 2.0 API may end up looking like:
https://github.com/fatcerberus/minisphere/blob/master/src/engine/pegasus.c
Title: Re: minisphere 3.3.0
Post by: N E O on June 16, 2016, 01:30:47 am
I can't believe I just put together that the screen is basically another surface. Internally is that how you treat drawing?
Title: Re: minisphere 3.3.0
Post by: Fat Cerberus on June 16, 2016, 01:36:38 am
Yes.  In Allegro the backbuffer gets an ALLEGRO_BITMAP pointer like all other textures, so it's easy to treat it the same as a surface.  Which is great, because it means you basically get render-to-texture in the API for free (just pass in a Surface object instead of screen).
Title: Re: minisphere 3.3.0
Post by: Fat Cerberus on June 20, 2016, 12:57:00 pm
So the result of all this, is that the Sphere 1.x API is being deprecated wholesale in minisphere 4.0, but will still be available, including map engine, so existing games will continue to run as before.  New games should instead use the new API exclusively, which is enforced by disabling the 1.x functions when a 2.0 manifest (game.json) is present.
Title: Re: minisphere 3.3.0
Post by: Fat Cerberus on June 24, 2016, 12:04:08 pm
For Color objects in Sphere 2.0, which scheme would be preferred for component properties:

Code: (javascript) [Select]

// Abbreviated components
color.r
color.g
color.b
color.a


-OR-

Code: (javascript) [Select]

// Sphere 1.x style
color.red
color.green
color.blue
color.alpha
Title: Re: minisphere 3.3.0
Post by: Radnen on June 24, 2016, 09:00:31 pm
I personally like smaller names, so I vote for r, g, b, a.
Title: Re: minisphere 3.3.0
Post by: Fat Cerberus on June 25, 2016, 03:13:16 am
Here's the documentation for the next-generation Sphere API:
https://github.com/fatcerberus/minisphere/blob/master/docs/spherical-api.txt
Title: Re: minisphere 3.3.0
Post by: Fat Cerberus on July 01, 2016, 03:14:37 pm
I decided to leave text rendering in the core.  RFN fonts can be loaded and rendered in script easily enough, but since I eventually want to support TTF fonts, there will need to be text drawing code in the engine anyway.

Progress report:
I started on the new JS map engine about a week ago.  Currently I can load .rts tilesets and draw individual tiles from them using Galileo.  I haven't done much more than that because I'm still trying to come up with a workable object model for it.  The main thing I keep getting hung up on is figuring out what the relationship should be between Map and Person objects.  Do we have it where Persons only have an X/Y value and are then added to the map they are on, or should the Person refer back to their containing map...?  Some combination of the two?  This is why I was so averse to the idea of an OO map engine, it's not always obvious how the different classes should relate to one another.

I guess one strategy would be to look at how other OOP systems with a high degree of interrelation (e.g. WinForms) handle cases like this.
Title: Re: minisphere 3.3.0
Post by: mezzoEmrys on July 02, 2016, 03:16:31 pm
Just glancing at the Person functions in the API on the wiki, I see a lot of potential for going either way. For the function "GetObstructingPerson", for instance, it could make sense for that to belong to the Person, in which case we'd have the Person go to the Map to find that information. If we put it in the Map, though, we end up keeping roughly the same signature for the function, where we have to pass the person we're asking about in.

I'd say that if you put functionality in Person for interacting with other Persons, Person would need the Map reference. However, the only Person-Person interaction seems to be obstruction and talking,  and obstruction also has tile obstruction. Unless you're planning to add more Person-Person interactions, it seems like just having Persons carry their own internal data and be added to the map they're on.
Title: Re: minisphere 3.3.0
Post by: N E O on July 02, 2016, 07:41:11 pm
Persons were associated with their initial containing maps unless they were set to persist_on_destroy_map, e.g. CreatePerson's last parameter. For v1 behavior keep it like this, but feel free to choose to keep entities independent of maps in the v2 redesign.
Title: Re: minisphere 3.3.0
Post by: Fat Cerberus on July 04, 2016, 01:45:36 am
One of the reasons I want to make entities independent objects is, it makes relocating entities at runtime a lot easier.  For example, teleportation.  In Sphere 1.x this is traditionally implemented by creating a persistent Person for the player character, and calling ChangeMap() to switch to the destination map.  The teleportation process is inseparable from the process of changing maps, which isn't always optimal.  In the system I'm envisioning, any entity can be relocated to a different map simply by changing its current location.  When a map is drawn, only those entities who are currently on that map will be rendered.

This system is nice because it allows not only the PC, but also other entities to be transported as well using the same mechanism.  You could allow NPCs to use doorways, for example, and the teleports would be fully functional for them.  In order to ensure the player character doesn't teleport out of existence, there should be a system where the person to whom the camera is attached is followed by the engine: If their location changes, the engine follows them to the new map.  This saves the duplicate effort of setting the PC's location and also changing maps, but also allows the camera to be refocused on any other active person at any time, even if they aren't on the same map.

Now I just have to stop being lazy and actually implement it... :P
Title: Re: minisphere 3.3.0
Post by: Radnen on July 04, 2016, 04:34:26 am
Are you saying if you focus the camera to a person on another map, we wold change maps? That would be very nifty. (Of course we can already code things around to do this in Sphere, but native support is always nice).
Title: Re: minisphere 3.3.0
Post by: Fat Cerberus on July 04, 2016, 01:22:03 pm
That's the idea, yeah: The camera concept is extended to not only encapsulate an X and Y value, but also which map you're on. So if the PC changes maps, the camera follows them automatically without having to call ChangeMap (or whatever the equivalent would be).

One situation where I think this will be really useful is cutscenes that switch between several maps, for example flashback sequences.  You could just place all the NPCs on the correct maps ahead of time and switch between them just by changing the camera focus.  Actual loading and unloading of maps could then be implicit, managed by the engine and saving the user from having to worry about it.
Title: Re: minisphere 3.3.0
Post by: Radnen on July 04, 2016, 06:30:11 pm
And it would be easier to track entities that move between rooms, like NPC's that enter and exit buildings based on set paths. Again, it is possible to do it right now, but it'll be a small coding nightmare to track it all on top of the engine since you cannot rely on the map engine entities.

Just to be clear: entities that exist on maps now no longer get destroyed, right? Despite the default behavior that they destroy when the map exists, they will stay in memory until you recall that map. I'm trying to wrap my ahead around how an analogue/persist like system plays into the new entities. We might have to add new triggers to an NPC, such as, map enter and map exit. That way we can move some behaviors from create/destroy that might now  not be invoked.
Title: Re: minisphere 3.3.0
Post by: Fat Cerberus on July 04, 2016, 06:48:47 pm
Yes, I think this system would necessitate entities remaining in existence until they are explicitly destroyed, say by calling person.dispose() or something.  They could then be part of the state saved in a gamesave, so that NPCs moved e.g. as part of a cutscene would remember their position when the file is reloaded.

If memory usage becomes an issue, it would of course be possible to load and unload the associated assets for an entity on demand.  This way you don't end up with hundreds of spritesets or whatever loaded simultaneously using a gig of RAM.
Title: Re: minisphere 3.3.0
Post by: DaVince on July 05, 2016, 02:58:28 pm
I really like the sound of all of this, frankly. :)
Title: Re: minisphere 3.3.0
Post by: Fat Cerberus on July 07, 2016, 11:59:30 am
The great thing is that it will be built on top of my threading framework (require("threads")) and Scenario (require("scenes")) so that will no longer need to be hacked in to work with the old-style update/render script system.  Everything will cooperate out of the box. :)
Title: Re: minisphere 3.3.0
Post by: Fat Cerberus on July 09, 2016, 01:53:23 am
Following up on the Scenario and threading integration, I'm thinking there will be no concept of a person command queue in the engine itself.  Persons themselves will simply have a basic set of actions they can perform: Change sprite direction, move one step in a direction, animate sprite.  These actions would be performed immediately (like calling QueuePersonCommand() with true as the last parameter), and any command queue-like functionality would instead be handled through Scenario.  This way Scenario won't feel "bolted on" like it does with the Sphere 1.x map engine (well, pretty much all 1.x map engine extensions are like that, it's just the nature of how it functions)
Title: Re: minisphere 3.3.0
Post by: mezzoEmrys on July 17, 2016, 12:30:41 am
Quick question, having built an SPK, how do I package the game to be executed as a standalone project, with customized settings, etc.?
Title: Re: minisphere 3.3.0
Post by: Fat Cerberus on July 17, 2016, 12:43:28 am
Not sure what you mean by customized settings, but minisphere associates itself with SPK files on installation, so you can just distribute that.  If you really want to distribute the engine with it, it should be enough to zip the SPK along with minisphere.exe and the system directory, just like you'd do with Sphere 1.x (minus the 8 or so DLL files).

To clarify: When you run minisphere.exe, if there's a single SPK file in the same directory as the engine, it will run that automatically.
Title: Re: minisphere 3.3.0
Post by: mezzoEmrys on July 17, 2016, 12:54:29 am
Customized settings like "not using fullscreen by default" or setting window size to scaled 800 by 600, etc.

Additionally, where are the saves for packaged games located (both on windows and linux)?
Title: Re: minisphere 3.3.0
Post by: Fat Cerberus on July 17, 2016, 01:13:25 am
Not sure how you changed the scaling, since minisphere is hardcoded to use 2x scaling for 400x300 and under and 1x for anything higher.  Unless you compiled the engine yourself?  As for disabling fullscreen, that's a Sphere Studio option, and not (currently) customizable for the engine (which in fact has no persistent settings).  The way you start the engine in windowed mode is to use the --window command-line option; this is what Sphere Studio does when you use that setting.

What I may do for minisphere 4.0, is add an optional setting in game.json which allows the game to specify whether it prefers to run in a window or fullscreen.  For a lot of games fullscreen is overkill.
Title: Re: minisphere 3.3.0
Post by: Fat Cerberus on July 17, 2016, 01:20:57 am

Additionally, where are the saves for packaged games located (both on windows and linux)?


For Sphere 1.x games, all writes to a packaged game are virtualized to [Documents]/minisphere/.spkcache.  This is the same for both Windows and Linux.  Note that for Sphere 2.0 games this won't be possible and you will have to use the ~/ SphereFS prefix.  But it'll be a few more months still before you have to worry about that. :)
Title: Re: minisphere 3.3.0
Post by: Fat Cerberus on July 28, 2016, 01:27:47 am
I did it: I got the Sphere 1.x and 2.0 APIs to coexist.  This means minisphere 4.0 will retain Sphere 1.x compatibility.  Yay!

Pulling this off required a few compromises I'm not too thrilled about, but it does avoid me having to rewrite half of the Spectacles codebase, so... well, can't win 'em all.  The cost of maintaining compatibility:


As for the non-interchangeable objects: While it's technically possible for the engine itself to treat, say, Sphere V1 and V2 Image objects equally (since they are internally the same type), it will never be possible for them to be interchangeable to JavaScript code as the methods and semantics vary.  So to me it made the most sense just to have the engine treat them as distinct and incompatible right out of the gate.  This way there's no confusion on users' part where old and new objects are compatible in some situations but not others.

So yeah, that's where I'm at right now.
Title: Re: minisphere 3.3.0
Post by: Fat Cerberus on July 28, 2016, 03:36:08 am
It occurs to me that the post above may have given the wrong impression.  While objects are not interchangeable, it's completely acceptable to mix Sphere 1.x and 2.0 code (even rendering code) in the same codebase.  You can even use Sphere 1.x drawing primitives (Rectangle() et al.) and then call screen.flip(), which will honor the current screen.frameRate.  Same for using Galileo in code that calls FlipScreen() and SetFrameRate().

What you can't do is stuff like this:

Code: (javascript) [Select]

var ss = LoadSpriteset('scott.rss');
var image = new Image('images/scott.png');  // Sphere 2.0 Image object
ss.images[0] = image;  // Error: expects a Sphere 1.x Image


Or this:

Code: (javascript) [Select]

var surf = CreateSurface(100, 100, Color.Black);  // Error again: Color.Black is a Sphere 2.0 Color object


Thus it can sometimes get awkward when using the classic map engine in Sphere 2.0 code because you pass Images and Colors around so much and it gets tempting to try to pass in the new-style objects you may already have on hand.
Title: Re: minisphere 3.3.0
Post by: DaVince on July 28, 2016, 04:57:29 pm
Damn, you're going through lengths to retain compatibility, and I have to say, that's really admirable of you to do. As well as clever, as it doesn't splinter the specific versions of Sphere someone needs in order to play Sphere games, old or new. :)
Title: Re: minisphere 3.3.0
Post by: Fat Cerberus on July 31, 2016, 02:38:52 am
New feature: The RNG object.  Unlike in previous versions where this was a singleton, in minisphere 4.0 it will be much more flexible.  You can create multiple generators and have full control over their state, allowing complex tricks to be done with the random number generator (time travel-like game mechanics, for example).  You could also use this to save and restore the state of the generator(s), to deter save scumming.

To illustrate:
Code: (javascript) [Select]

var gen = RNG.fromSeed(812);
var state = gen.state;
var x = gen.next();  // ~0.028
var y = gen.next();  // ~0.758
var gen2 = RNG.fromState(state);  // you could also set gen.state directly
var z = gen2.next();  // ~0.028 (!)
var w = gen2.next();  // ~0.758 (!!)


This is a very low-level API which gives you direct access to the underlying generator (xoroshiro128+).  All the convenience methods from previous versions like RNG.uniform and RNG.string have been moved into a JS module "random" that you load using require("random").
Title: Re: minisphere 3.3.0
Post by: Radnen on July 31, 2016, 03:03:43 am
By default does it use the clock? Or do you have to put a seed each time? Just curious. Also is the state the same thing as returning the 812? Because to me it seems redundant when you could just store the seed and reuse it, unless there's more going on behind the scenes.
Title: Re: minisphere 3.3.0
Post by: Fat Cerberus on July 31, 2016, 03:23:28 am
There is RNG.Default, which is seeded from the clock, and `new RNG()` will likewise use the clock as a seed.

The seed and state are different things.  The seed (812 in this case) determines the specific sequence that will be generated and is just a number.  The state is a 32-character hexadecimal string you can use to resume the generator from exactly where you left off (to prevent save scumming, this is what you'd store in the save file).  You can't do this with the seed alone because the seeding algorithm is not reversible.

Note that if I had stuck with Mersenne Twister, the state would have been a 2.5KB string!
Title: Re: minisphere 3.3.0
Post by: Fat Cerberus on July 31, 2016, 11:45:46 am
To illustrate:
Code: (javascript) [Select]

var gen = RNG.fromSeed(812);
console.log(Math.floor(gen.next() * 10));
console.log(Math.floor(gen.next() * 10));
var state = gen.state;
console.log("state saved:", state);
console.log(Math.floor(gen.next() * 10));
console.log(Math.floor(gen.next() * 10));
console.log(Math.floor(gen.next() * 10));
gen.state = state;
console.log("state loaded:", gen.state);
console.log(Math.floor(gen.next() * 10));
console.log(Math.floor(gen.next() * 10));
console.log(Math.floor(gen.next() * 10));
system.exit();


Output:
Code: [Select]

C:\src\spectacles-i>ssj -r dist
SSJ 4.0a1 Sphere JavaScript debugger (x64)
a powerful JavaScript debugger for minisphere
(c) 2015-2016 Fat Cerberus

starting C:/src/spectacles-i/dist/... OK.
connecting to 127.0.0.1:1208... OK.
verifying... OK.
querying target... OK.
    game: Spectacles: Bruce's Story
    author: Author Unknown

log: 0
log: 7
log: state saved: 19623bbbaffade049a164924bc152c80
log: 5
log: 1
log: 2
log: state loaded: 19623bbbaffade049a164924bc152c80
log: 5
log: 1
log: 2
debugger disconnected normally.
SSJ session terminated.
Title: Re: minisphere 3.3.0
Post by: Fat Cerberus on August 02, 2016, 12:29:40 pm
Here's the plan: minisphere 4.0 will be an experimental release, to test the waters for the Sphere v2 API.  There are a few areas that might be rough around the edges or in need of tweaks; I'm not ready to commit to the exact setup just yet.  minisphere 4.x will thus have the Sphere API level set to 0, specifically reserved to mean "unstable".  If that's successful and everyone is happy with what's available, minisphere 5.0 will have the API level bumped up to 1, finalizing the API.  From there on, features will only be added (and the API level bumped) in a backwards compatible manner.  Alternatively, features can be added first as extensions, and only later incorporated into the core API if they prove useful.

Doing it this way should also accelerate the 4.0 release, since I don't have to make sure the API is perfect before the release.  If breaking changes need to be made during the 4.x cycle, that can be done freely.
Title: Re: minisphere 3.3.0
Post by: Fat Cerberus on August 05, 2016, 11:26:29 am
Beta 1 for minisphere 4.0 is available.  Install it and try out the new API. ;)
Title: Re: minisphere 4.0.0
Post by: Fat Cerberus on August 12, 2016, 12:27:01 am
minisphere 4.0 is out.  The 4.x.x series will be a rolling release to get the toolchain (SSJ, Cell, possible upcoming CLI tools) up the speed before finalizing the Sphere v2 API.  I could have delayed the 4.0 release further, but I figured it was better to get Sphere v2 out the door now so that the new API is available to everyone as soon as possible.

This has been a long time coming, now go crazy! :D
Title: Re: minisphere 4.0.1
Post by: Fat Cerberus on August 14, 2016, 02:09:30 am
First update in the minisphere 4.0 rolling release cycle: minisphere 4.0.1 updates system.extensions to allow you to test for the presence of an extension using a simple if statement:

Code: (javascript) [Select]

// while building save game data:
if (system.extensions.sphere_stateful_rng) {
    saveData.rngState = monsterDropRNG.state;
}


Extension strings returned in this release are:

Title: Re: minisphere 4.0.1
Post by: FBnil on August 14, 2016, 04:14:14 pm
downloaded. Will test it in the afternoons...
Title: Re: minisphere 4.0.1
Post by: Eggbertx on August 18, 2016, 08:58:58 pm
Ooh, fancy new API. When do you plan on removing Sphere 1.x API compatibility, or is that not going to happen?
Title: Re: minisphere 4.0.1
Post by: Fat Cerberus on August 18, 2016, 09:06:27 pm
Not for the foreseeable future.  I know there are existing large Sphere v1 codebases (Radnen's Blockman, e.g.) and I want to make migration as simple as possible.  For this reason it's perfectly acceptable to mix v1 and v2 function calls.

That said, the v1 API should be considered fully deprecated by this release and all new code should be written to the Sphere v2 API.  The legacy code is not likely to receive much maintenance going forward, except for bug fixes.
Title: Re: minisphere 4.0.1
Post by: Eggbertx on August 18, 2016, 09:46:53 pm
I figured that was the case, I just wanted to make sure that it wouldn't be completely removed.
Also, I haven't checked with the newest release, but I noticed minisphere as of 3.3 at least doesn't seem to take into consideration other possible game directories when GetGameList() is used. For example, I have minisphere installed in C:\Program Files\minisphere, but all of my games are in my dropbox folder with Sphere 1.5 and 1.6. Since I have Sphere Studio set to look in that directory, is it possible for minisphere to do the same somehow?
Title: Re: minisphere 4.0.1
Post by: Fat Cerberus on August 18, 2016, 09:54:29 pm
Technically all that's required for v1 compatibility for GetGameList() is to look in <engine_dir>/games, which minisphere does.  It also looks in <documents>/Sphere Studio/Projects, since that's where Studio puts new games by default.  Custom directories is a bit harder to support - the engine is self-contained and has no knowledge of the outside world, so to speak.

I'll see what I can do though, if anything.
Title: Re: minisphere 4.0.1
Post by: Eggbertx on August 19, 2016, 12:06:54 am
So should I just use symlinks or something? I wish it was as easy to do in Windows as it is in Linux.
Title: Re: minisphere 4.0.1
Post by: Fat Cerberus on August 19, 2016, 12:39:18 am
That should work, yeah.  The command you're looking for is

Code: [Select]
mklink /d "C:\Program Files\minisphere\games" <target_dir>


You need to do that in an elevated command prompt though, otherwise you won't be able to create a symlink inside Program Files.
Title: Re: minisphere 4.0.1
Post by: Eggbertx on August 19, 2016, 12:41:48 am
I've used it once or twice before. And it requires elevated privileges regardless of location.
Title: Re: minisphere 4.1.0
Post by: Fat Cerberus on August 22, 2016, 02:49:57 am
minisphere 4.1.0 is available now and introduces the new "assert" module.  This module has the same interface as the Node.js module by the same name:
https://nodejs.org/api/assert.html

Unlike ssj.assert() which only has an effect with an attached debugger and doesn't change program flow (beyond optionally triggering a breakpoint), the assert module methods will throw an AssertionError if a check fails.  This makes them suitable for argument validation and the like:

Code: (javascript) [Select]

const assert = require('assert');

function feedFace(what, howMuch)
{
    assert.equal(typeof what, 'string');
    assert.equal(typeof howMuch, 'number');

    /* ...carry on */
}
Title: Re: minisphere 4.1.1
Post by: Eggbertx on August 24, 2016, 01:40:50 am
Did you just update twice tonight?
Title: Re: minisphere 4.1.1
Post by: Fat Cerberus on August 24, 2016, 01:46:33 am

Did you just update twice tonight?


I put up minisphere 4.1.0 two days ago (8/22).  This is 4.1.1, released 8/24.  I do tend to be trigger happy with hotfixes, though. :)
Title: Re: minisphere 4.1.1
Post by: Eggbertx on August 24, 2016, 01:49:23 am
Oh, alright. I cloned the git repo a little while ago to update minisphere to 4.1.0 in the AUR, and then got an email less than five minutes after I updated the AUR package that you had put out 4.1.1
Title: Re: minisphere 4.1.1
Post by: Fat Cerberus on August 24, 2016, 02:29:36 am

Oh, alright. I cloned the git repo a little while ago to update minisphere to 4.1.0 in the AUR, and then got an email less than five minutes after I updated the AUR package that you had put out 4.1.1


Thanks for updating that package.  I had forgotten all about it.  Ubuntu PPA packages for 4.1.1 are already up.

Note for Windows users: If you had any version of minisphere prior to 4.0.0 installed at any point, I recommend uninstalling the engine, deleting C:\Program Files\minisphere completely, then reinstalling fresh.  Some things got moved around in 4.x that may cause issues/confusion if the old files aren't removed.  Removing outdated files when updating an already-installed app isn't one of Inno Setup's strong suits. :P
Title: Re: minisphere 4.1.1
Post by: Eggbertx on August 24, 2016, 03:05:54 am
I kept going between forgetting about it and procrastinating, because it was a bit of trouble getting it started. But now all I have to do is change the version number and update the checksum for the source archive in two files and push them to the git repo.
Title: Re: minisphere 4.1.1
Post by: Eggbertx on August 24, 2016, 08:20:11 pm
While trying to figure out why Sphere's v2 API wasn't running anything (after I created game.json), I noticed that there's a syntax error in your example JSON in sphere2-api.txt
Quote from: javascript

    {
        "name": "Spectacles: Bruce's Story",
        "author": "Fat Cerberus",
        "summary": "Follow Scott Starcross in his quest to stop the Primus.",
        "resolution": "320x240",
        "main": "main.js",
    }


When I tried to run it from the Sphere IDE, the console would open and immediately close without any warnings or errors. I had to open Windows' comand line externally and run it from there to see the error message.

Also, does the Sphere v2 API have no entry point function?
Title: Re: minisphere 4.1.1
Post by: Fat Cerberus on August 24, 2016, 08:38:03 pm

While trying to figure out why Sphere's v2 API wasn't running anything (after I created game.json), I noticed that there's a syntax error in your example JSON in sphere2-api.txt


Thanks, I'll fix the documentation and maybe add a warning message in Sphere Studio when the engine fails to load for any reason.

Quote

Also, does the Sphere v2 API have no entry point function?


This is an area where the documentation needs to be improved, but starting with minisphere 4.1.1, when you use a JSON manifest, the engine runs in native Sphere v2 mode.  Among other things, this means that your main script is loaded as a CommonJS module, like in Node.JS.  Any startup code should be placed at the top level, outside of any functions, and variable or function declarations are local to the module unless you explicitly make them global:

Code: (javascript) [Select]

global.myGlobalVar = 812;


If you add "version": 1 to the manifest, the engine runs in compatible mode, the same as if you had a game.sgm file.  In compatible mode the main script runs as global code and the engine calls the game() function, just like Sphere 1.x.
Title: Re: minisphere 4.1.1
Post by: Fat Cerberus on August 25, 2016, 03:11:44 am
Here's the improved documentation for the manifest format:
https://github.com/fatcerberus/minisphere/blob/master/docs/sphere2-api.txt#L15-L82

The edits will be in the next release.
Title: Re: minisphere 4.1.1
Post by: Eggbertx on August 26, 2016, 05:20:54 pm
I'm not sure at what point this started happening, but it looks like on Linux, minisphere silently doesn't play some IT tracker files and throws errors for others.
Also the map engine doesn't seem to run. It just ignores calls to MapEngine(file,framerate). But running 1.5 through wine shows that it should be working correctly. (And of course it does in Windows)
(http://imgur.com/iQgmt7dl.png)

I'm not sure if this is entirely related, but when I tried to use Windows minisphere through wine to get the above screenshot, calling MapEngine caused it to crash.

using SSJ, I got this output:
Code: [Select]

waiting for connection from debug client...
connected to debug client at 127.0.0.1
Segmentation fault (core dumped)
Title: Re: minisphere 4.1.1
Post by: Fat Cerberus on August 27, 2016, 12:12:32 am
Hm, those map engine issues are weird.  I can't reproduce them.  minisphere 4.1.1 runs Spectacles just fine, and that uses the 1.x map engine.  I can also run Radnen's Blockman and DaVince's Sir Boingers just fine without issue.  So I'm not sure what's going on for you.  I'll look into it, though.
Title: Re: minisphere 4.1.1
Post by: Eggbertx on August 27, 2016, 04:59:31 pm
I'll make two new projects then, one will use the 1.5 game.sgm system and the other will use the new game.json system. Both will have the IT modules I tried (and maybe some others) and will test the map engine. And of course I'll test them in Linux and Windows. I sold my macbook so I can't test OS X though.

Also, could you make errors in general more specific? I know the modules were there so it's not an issue of it not finding the file, but it would be nice to get more detailed info on what's going wrong.
Title: Re: minisphere 4.1.1
Post by: Eggbertx on August 27, 2016, 05:38:51 pm
Here is a picture (not a great one, but enough to be able to see) showing my laptop running Linux and my PC running Windows playing the same game. My Sphere games folder is in my Dropbox folder, so both computers are playing the exact same version. The PC has no problem playing the IT module and running the map engine, but Linux isn't loading the module.

When I remove the lines that load and then play the module, it works just fine on both.

Edit: After looking up a tutorial on audio in Allegro, it looks like it's an issue with Allegro, not minisphere. It plays ogg and wav files fine, but not IT. After installing the dumb (http://dumb.sourceforge.net/) package, it caused my player to segfault, but there is no change in minisphere. I submitted an issue to the GitHub repo, hopefully they'll be able to give me an answer as to what's going on.
Title: Re: minisphere 4.1.1
Post by: Fat Cerberus on August 27, 2016, 11:35:12 pm
The Windows builds of minisphere are statically linked with Allegro and all related dependencies (including DUMB), so that's probably why installing that package didn't change anything.  On the other hand Ubuntu builds are dynamically linked; I assume the Arch Linux packages are too?

Regarding the error messages: If you mean the "unable to load sound" error, I don't know that it's possible for that to be more specific.  One of the shortcomings of Allegro is that it doesn't provide error codes in its API.  minisphere can only ask Allegro to attempt loading the module file, and it will either succeed (and I get a valid stream pointer back), or fail (and I get a NULL back).

So just to clarify: Do you mean that the MapEngine() crash doesn't happen if you switch out the IT module for a .wav or .ogg file?  If that's the case, then it looks like the issue is indeed out of my hands.  I did see your bug report on the Allegro repo, by the way; it might help them if you provide an .it file that reproduces the segfault.
Title: Re: minisphere 4.1.1
Post by: Eggbertx on August 28, 2016, 12:28:37 am
Most AUR packages just download the source and build/install it like you normally would manually, so minisphere installed in Arch shouldn't be any different from Minisphere installed in Ubuntu.

So Allegro doesn't have anything similar to SDL's SDL_GetError()/IMG_GetError()/Mix_GetError()/etc?

And I must have been mistaken, I thought it was the map engine, but it looks like it's just the tracker file.
Title: Re: minisphere 4.1.1
Post by: Fat Cerberus on August 28, 2016, 04:30:37 pm
It's weird that you should have issues with IT modules.  minisphere runs Blackfoot, which uses IT modules, nearly flawlessly (save for a rogue mp3 sound, minisphere doesn't support mp3).  In fact that game was my first compatibility target when I started development. :)

It must only be certain modules that Allegro has trouble with.
Title: Re: minisphere 4.1.1
Post by: Eggbertx on August 28, 2016, 06:21:50 pm
After submitting a bug report to Allegro's GitHub repo, they suggested that it might have something to do with the Arch package, so I submitted a bug report to the Arch package.
Title: Re: minisphere 4.1.1
Post by: N E O on August 28, 2016, 08:08:59 pm
Re DUMB - are you using kode54's updated fork of DUMB (https://github.com/kode54/dumb) as the base?
Title: Re: minisphere 4.1.1
Post by: Fat Cerberus on August 31, 2016, 12:56:41 pm
Having recently purchased an Xbox One controller for my new gaming PC (custom built!), I found a longstanding bug in the map engine - it doesn't recognize joystick input for movement.  It responds to pressing the Talk button, but not the analog stick.  I'll fix it in the next update.
Title: Re: minisphere 4.1.1
Post by: Eggbertx on August 31, 2016, 03:01:50 pm

Re DUMB - are you using kode54's updated fork of DUMB as the base?

I'm not, just what's in the Arch Linux repo.



Having recently purchased an Xbox One controller for my new gaming PC (custom built!), I found a longstanding bug in the map engine - it doesn't recognize joystick input for movement.  It responds to pressing the Talk button, but not the analog stick.  I'll fix it in the next update.

Is the left analog stick going to be treated the same way as the dpad, or is it going to be its own thing (so you can walk fast or slow)?
Title: Re: minisphere 4.1.1
Post by: Fat Cerberus on August 31, 2016, 05:46:38 pm
The d-pad is treated as a POV-hat by the DirectInput drivers, so it's a separate thing from the analogs.  That said, the Sphere 1.x map engine just uses a single movement speed, analog input is converted to digital with a dead zone of ~50%.

It will be possible to support true analog input with varying speed in the new map engine that I eventually want to get around to making. ;)
Title: Re: minisphere 4.1.1
Post by: Eggbertx on August 31, 2016, 06:54:06 pm
I figured that would be the case, and while I doubt the analog speed will be used much, if the outcome is worth the effort, I'm sure someone might have a use for it.

Edit: There's an issue with your xml parser, and possibly some other modules as well. When I run the following code
Code: (javascript) [Select]

const xml = require("xml");

function game() {
    ml = xml.load("sample.xml");
}


Just clicking "Test" gives me the error "ReferenceError: 'FileStream' undefined" in #/modules/xml.js:17.
If I use the debugger, I see the exception "ReferenceError: module not found `stream`" in #/modules/lib/sax.js:163
Title: Re: minisphere 4.1.1
Post by: Fat Cerberus on August 31, 2016, 11:42:02 pm
That last ReferenceError you mentioned is harmless.  sax.js is trying to load the built-in Node.js "stream" module, which of course fails.  The error is caught and sax uses a fallback.  SSJ always reports thrown errors, even if they are caught.  If you're using Sphere Studio, caught errors will have a green checkmark next to them.

The FileStream error I do have to fix, though.  Originally, with Sphere v2, you opened files like this:
Code: (javascript) [Select]

var file = new FileStream(fileName, 'r');


However, since I wrote the xml module that has changed to:
Code: (javascript) [Select]

var file = fs.open(fileName, 'r');


It seems that I never updated the xml module to reflect that, oops!
Title: Re: minisphere 4.1.1
Post by: Eggbertx on August 31, 2016, 11:49:51 pm
So until it's officially updated, can I just replace it myself?
Title: Re: minisphere 4.1.1
Post by: Fat Cerberus on August 31, 2016, 11:54:18 pm
Yeah, that should work.
Title: Re: minisphere 4.1.1
Post by: Fat Cerberus on September 12, 2016, 03:24:31 am
The new joystick API coming in minisphere 4.2 will allow you to easily query connected input devices using Link, like this:

Code: (javascript) [Select]

const link = require('link');

// historically, Sphere requires a gamepad with at least 2 axes (X/Y) and
// 5 buttons (A, B, X, Y, Start) for full operation.
var devices = link(Joystick.getDevices())
.where(function(it) { return it.numAxes >= 2; })
.where(function(it) { return it.numButtons >= 5; })
.toArray();


This allows minisphere to support any number of gamepads while at the same time allowing games to ignore any devices without enough inputs.  While that latter thing was technically possible in Sphere 1.x, the engine itself would only tell you about 4 devices max.

Not that I can see too many people hooking up 10 gamepads to their computer, but the API flexibility is there to allow it if they do. :P
Title: Re: minisphere 4.2.0
Post by: Fat Cerberus on September 14, 2016, 02:37:19 am
minisphere 4.2 is out, bringing with it the brand-new joystick API, a new "joy" module, and basic support for TextEncoder and TextDecoder (utf-8 only for now).  There are also a few bug fixes: the XML module works again (thanks Eggbert for finding this), and a long-standing bug in the map engine which prevented the input person from being moved with the joystick is now fixed.  Refer to sphere2-api.txt and miniRT-api.txt for documentation on the new objects and functions.

No more new features are planned for minisphere 4.x.  The remainder of the 4.x cycle will instead be spent polishing up the Sphere v2 API and getting it ready to be finalized in minisphere 5.0.  At that point system.apiLevel will be set to 1 and no more incompatible changes to the API will be allowed.
Title: Re: minisphere 4.2.0
Post by: DaVince on September 14, 2016, 11:14:36 am
I swear one of these days I'm going to actually make use of my One Game A Month account, and get all familiar with the New And Improved Sphere... Things are starting to look really sweet.

One thing I keep thinking about nowadays is cross-platform compatibility. By which I mean phones or HTML5. Do you think this could ever feasibly happen with how minisphere has changed things? (Just curious. I'd love to use it that way. "mobile" support could partially be an extra module to add in the support for eg. a controller overlay.)
Title: Re: minisphere 4.2.0
Post by: Fat Cerberus on September 14, 2016, 11:40:38 am
I really want to do at least mobile support.  The web is a totally different beast since everything is async, but a Sphere app certainly seems feasible.  I can technically compile Android apps using Visual Studio, but I've been too lazy to learn how everything works.  I'm pretty sure my next phone is going be Android-based though (was going to be an iPhone 7, but I hate the changes in the iOS 10), so maybe that'll light a fire under me to make Mobile Sphere work. :P
Title: Re: minisphere 4.2.0
Post by: DaVince on September 15, 2016, 08:17:00 am
That's good to hear. :) Nice work on all the progress, by the way. I swear I'll eventually get around to making something with it... :P
Title: Re: minisphere 4.2.0
Post by: Eggbertx on September 16, 2016, 02:51:51 pm
I updated the AUR package
Title: Re: minisphere 4.2.0
Post by: Fat Cerberus on September 16, 2016, 03:40:44 pm

I updated the AUR package


Good to hear, thanks. :D

Edit: The AUR page claims to still be using the tarball for 4.1.1?
Title: Re: minisphere 4.2.0
Post by: Eggbertx on September 16, 2016, 03:43:59 pm
I kinda wish Github had a way to show the release download's checksum. That way I wouldn't have to download it just to check, since AUR packages require it in the PKGBUILD and .SRCINFO metadata files.
Title: Re: minisphere 4.2.0
Post by: Fat Cerberus on September 17, 2016, 12:47:36 am
@Eggbert: Just double-checked, I was right.  Your "4.2.0" AUR package is actually using sources for 4.1.1.  You might want to fix that. ;)
Title: Re: minisphere 4.2.0
Post by: Eggbertx on September 17, 2016, 05:19:55 pm
It shouldn't be. I just checked and it it downloads, builds, and installs the 4.2 sources.
minisphere PKGBUILD (https://aur.archlinux.org/cgit/aur.git/tree/PKGBUILD?h=minisphere)
minisphere .SRCINFO (https://aur.archlinux.org/cgit/aur.git/tree/.SRCINFO?h=minisphere)
Title: Re: minisphere 4.2.0
Post by: Fat Cerberus on September 17, 2016, 06:19:50 pm
Huh.  Maybe the AUR site is bugged or something then.  See the screenshot below, under Sources it lists the tarball for 4.1.1:
https://drive.google.com/open?id=0BxPKLRqQOUSNRGFGWGw0OEJ4bDg

If I download that tarball, it is indeed the 4.1.1 sources.  Weird.

Edit: The .SRCINFO you linked points to the 4.1.1 tarball.
Title: Re: minisphere 4.2.0
Post by: Eggbertx on September 17, 2016, 06:31:42 pm
Oh that's right, I forgot the .SRCINFO doesn't use variables.
Title: Re: minisphere 4.2.1
Post by: Fat Cerberus on September 21, 2016, 03:02:16 am
minisphere 4.2.1 renames a few APIs for consistency.  Notably, the "terminal" module is now called simply "term"
Title: Re: minisphere 4.2.1
Post by: Eggbertx on September 21, 2016, 02:44:40 pm
Should I update the AUR package or wait?
Title: Re: minisphere 4.2.1
Post by: Fat Cerberus on September 21, 2016, 02:51:49 pm
You don't have to, it's not an important update.  No bug fixes or anything like that.
Title: Re: minisphere 4.2.1
Post by: Eggbertx on September 25, 2016, 05:03:40 pm
For some reason the AUR package allegro5-git, which uses allegro 5.1.11 (while the latest version is 5.2.1.1) can play tracker files.
Title: Re: minisphere 4.2.2
Post by: Fat Cerberus on September 28, 2016, 12:28:56 am
minisphere 4.2.2 fixes the "struct" and "logger" modules, which stopped working due to the recent changes to the FileStream object.

@Eggbert: You should update the AUR package, since the broken modules in question are part of the standard library (miniRT).
Title: Re: minisphere 4.2.2
Post by: Eggbertx on September 29, 2016, 02:54:56 pm
Updated. Also, I'm not sure how relevant or useful it is, but minisphere compiles with clang with a few noncritical errors. It might be worth looking into to see how it compares in speed and optimization.
Title: Re: minisphere 4.2.2
Post by: Fat Cerberus on September 29, 2016, 07:25:43 pm
Yeah, there's a clang build in minisphere's Travis test.  Most of the warnings are because minisphere does this a lot:

Code: (c) [Select]

if (something = something_new(...)) {
    // ...
}


Clang complains because I'm doing assignment directly in the conditional but it thinks I want to do comparison instead.  I could make the warnings go away by adding an extra set of parentheses around the assignment, but I'm too lazy. :P
Title: Re: minisphere 4.2.2
Post by: Fat Cerberus on October 03, 2016, 02:28:11 am
See here:
https://github.com/svaarala/duktape/pull/987

I have a Duktape patch in the works to add support for the new JS exponentiation operator (x**y), hopefully that'll make it into minisphere soon. :D
Title: Re: minisphere 4.2.2
Post by: DaVince on October 03, 2016, 07:43:21 am
Huh, never expected a new operator like that to enter JS!
Title: Re: minisphere 4.2.2
Post by: Eggbertx on October 03, 2016, 01:07:18 pm
I've never heard of that, but I always did wonder why there isn't an exponent operator.
Title: Re: minisphere 4.2.2
Post by: Fat Cerberus on October 03, 2016, 03:55:09 pm

I've never heard of that, but I always did wonder why there isn't an exponent operator.


It's brand new in the latest version of the JavaScript standard (ECMA 7), and yes, it's quite useful especially for games where you do a lot of squaring.

C and C++ don't have an exponent operator either, which always annoyed me having come from a QuickBasic/VB background initially.  For a while I wondered why x ^ 2 in C wasn't giving me the expected results... :P
Title: Re: minisphere 4.2.2
Post by: Flying Jester on October 03, 2016, 04:39:04 pm
I had always expected it would be ^^. But I guess this means I can still dream of a day where ^^ will mean logical-xor.
Title: Re: minisphere 4.2.2
Post by: N E O on October 04, 2016, 03:57:21 pm
I guess JS-ers were expected to use Math.pow all the time?

Also, is anyone here familiar with creating a port/pkg on FreeBSD? Once I get minisphere compiled properly on it I'd like to see how far we can take that, though admittedly we'd probably have to nudge Allegro into updating their BSD port to 5.2.x to cover our bases.
Title: Re: minisphere 4.2.2
Post by: Fat Cerberus on October 04, 2016, 05:11:48 pm

I guess JS-ers were expected to use Math.pow all the time?


Yeah, which is annoying because it makes code more difficult to read, as well as require up to two unnecessary property reads (Math -> Math.pow) plus a JS function call.  It's difficult to optimize, especially for interpreted engines, since the bytecode compiler has to know ahead of time that `Math.pow` hasn't been reassigned.  With a proper operator the compiler can easily see an expression like 2 ** 16 and do constant folding on it.

C also has the same issue with pow(), but it's not as big a deal there because pow(base, exp) adds less visual noise compared to Math.pow(base, exp).
Title: Re: minisphere 4.2.2
Post by: Flying Jester on October 04, 2016, 06:58:20 pm

Also, is anyone here familiar with creating a port/pkg on FreeBSD? Once I get minisphere compiled properly on it I'd like to see how far we can take that, though admittedly we'd probably have to nudge Allegro into updating their BSD port to 5.2.x to cover our bases.


I'm pretty sure the normal way to do it is to create an entry in the ports tree and use the infrastructure there to create the package.
Title: Re: minisphere 4.2.3
Post by: Fat Cerberus on October 06, 2016, 03:06:52 am
minisphere 4.2.3 is available now.  It adds the new JS exponentiation operator and improves the Encoding API to more closely match the WHATWG encoding standard.  Specifically, TextEncoder now outputs pure UTF-8 always, rather than copying the string byte-for-byte from Duktape; and TextDecoder now supports the "fatal" and "ignoreBOM" flags.
Title: Re: minisphere 4.2.4
Post by: Fat Cerberus on October 07, 2016, 02:53:30 am
Oops!  I stumbled on a nasty regression in 4.2.3 related to text validation that could cause Unicode text to become corrupted internally, leading to subtle bugs.  While I was at it I improved the TextDecoder implementation - it now supports streaming.
Title: Re: minisphere 4.2.4
Post by: Fat Cerberus on October 07, 2016, 02:53:04 pm
I just implemented a new DataReader object to replace the methods removed from FileStream in v4.2:
https://github.com/fatcerberus/minisphere/blob/master/docs/miniRT-api.txt#L514-L615

Not sure if we also need a DataWriter - it seems that going forward JSON is a better choice for an output format.

edit: Ended up implementing a DataWriter class after all.  Even if it's not used for writing binary files, it can still be useful to send data over a socket.
Title: Re: minisphere 4.2.4
Post by: Eggbertx on October 13, 2016, 02:55:28 pm
Does minisphere not use ~/minisphere//games for the games directory in Linux any more? I have ~/minisphere/games/ as a symlink to ~/Dropbox/Programming/Sphere/games, and the default startup game says there aren't any games.

Also, until further notice I'm replacing the AUR package's allegro dependency with allegro5-git. For whatever reason, the AUR package can play tracker files. It may or may not just be an issue with my setup, but I don't know what else to do, considering it conflicts with the official allegro package.
Title: Re: minisphere 4.2.4
Post by: Fat Cerberus on October 13, 2016, 05:28:39 pm
~/minisphere/games should still be correct.  No idea what's wrong there.

Re: Allegro, that was actually answered on your Github issue.  The official package doesn't include DUMB because for some reason the DUMB maintainer has it dependent on Allegro 4.  So there's a conflict there.  I guess the AUR package doesn't have that issue because it can use unofficial versions of dependencies.
Title: Re: minisphere 4.2.4
Post by: Eggbertx on October 13, 2016, 08:19:37 pm
Oh, I didn't notice those last two replies
Title: Re: minisphere 4.2.4
Post by: Fat Cerberus on October 14, 2016, 11:03:02 pm
@Eggbert:
Checked into your game directory issue.  Search paths are here:
https://github.com/fatcerberus/minisphere/blob/master/src/engine/vanilla.c#L1045-L1046

homepath() points here:
https://github.com/fatcerberus/minisphere/blob/master/src/engine/utility.c#L31

The correct path on Ubuntu is therefore ~/Documents/minisphere/games.  I'm not sure where ALLEGRO_USER_DOCUMENTS_PATH points on Arch, you'd have to look into that.
Title: Re: minisphere 4.2.4
Post by: Eggbertx on October 20, 2016, 02:47:14 pm
Alright, I fixed it, I just thought it was kind of weird, since I could have sworn it was ~/minisphere before. I moved it to ~/Documents/minisphere and everything is fine now.

Also, I've been meaning to ask. From sphere2-api.txt
Quote

new Font(filename);

    Constructs a Font object from a font file.  Currently only the Sphere RFN
    font format is supported.


Do you have plans to support TTF fonts?
Title: Re: minisphere 4.2.4
Post by: Fat Cerberus on October 21, 2016, 10:11:48 am

Also, I've been meaning to ask. From sphere2-api.txt
Quote

new Font(filename);

    Constructs a Font object from a font file.  Currently only the Sphere RFN
    font format is supported.


Do you have plans to support TTF fonts?


Yes, I just haven't thought of a good API design for it yet.  TrueType has fancy features that raster fonts don't have to worry about like kerning, etc.  TTF also means full support for Unicode, and minisphere isn't fully Unicode-aware yet, there's only enough internal Unicode support to deal with Duktape strings which are UTF-8 encoded and need to be converted to CP-1252 for compatibility with Sphere 1.x (that was quite fun to fix a while ago, as DaVince can attest to :P).
Title: Re: minisphere 4.2.4
Post by: Fat Cerberus on October 21, 2016, 10:53:19 am
I'm experimenting with using SpiderMonkey in minisphere instead of Duktape.  So far I built the latest SpiderMonkey source from Mercurial, and... well, not much else.  I'll first have write a C binding for SM before I can do anything with it because it now has a C++-only API and the entire minisphere codebase is C.  So that'll be time-consuming.  Let's see how this project goes. :)
Title: Re: minisphere 4.2.4
Post by: Fat Cerberus on October 21, 2016, 07:55:11 pm
Hm, Spidermonkey seems to be going down V8's path.  The API can change arbitrarily between releases, quickly enough that the documentation is not always kept up to date (there's a tutorial on MDN that doesn't even compile because it doesn't reflect the current API), and I found this note in the release notes for SM 45:

Quote
Mozilla has no plans to keep the JSAPI stable for embedders. We have instead chosen to concentrate on overall performance and accuracy as our primary concerns.


Lovely.  So that should be a fun learning curve if I can't even trust Mozilla's own API documentation to be kept up to date.  Thank goodness for intellisense. :P
Title: Re: minisphere 4.2.4
Post by: Eggbertx on October 22, 2016, 04:06:37 pm

Hm, Spidermonkey seems to be going down V8's path.


I thought V8 was supposed to be relatively friendly with embedded projects (for example, node.js)
Is that no longer the case, or is the node.js development team willing to deal with the changes in the API?
Title: Re: minisphere 4.2.4
Post by: Fat Cerberus on October 22, 2016, 04:52:42 pm
Back when FJ was working on TurboSphere the V8 API was a moving target and in a constant state of flux, I don't know if it's stabilized since then.  That's one of the things that ultimately led him to go with SpiderMonkey instead. (The other reason being that V8's garbage collector is too lax leading to native resources not being freed in a timely manner)
Title: Re: minisphere 4.2.4
Post by: Eggbertx on October 22, 2016, 08:11:54 pm
Ah, well that stinks. So with those two out, does that make Duktape our best option, for better or worse?

Also it's not a huge issue but I noticed I get a parse error if the last line of a script is a comment.
Title: Re: minisphere 4.2.4
Post by: Fat Cerberus on October 22, 2016, 11:16:46 pm

Ah, well that stinks. So with those two out, does that make Duktape our best option, for better or worse?

Also it's not a huge issue but I noticed I get a parse error if the last line of a script is a comment.


For now, probably yes.  I'm still going to keep experimenting with SpiderMonkey though, but it'd be a pretty big refactoring project to actually make the switch.  Most likely minisphere 5.0 will still use Duktape.

I'll look into that comment issue and submit a bug report if needed.
Title: Re: minisphere 4.2.4
Post by: Flying Jester on October 22, 2016, 11:23:33 pm
More than just being less stable, V8's API isn't very friendly. It's developed solely to be a part of Chrome, while SpiderMonkey is intended to be a good JS VM. As a part of that, it is also far better documented.
Title: Re: minisphere 4.2.4
Post by: Fat Cerberus on October 22, 2016, 11:33:12 pm

More than just being less stable, V8's API isn't very friendly. It's developed solely to be a part of Chrome, while SpiderMonkey is intended to be a good JS VM. As a part of that, it is also far better documented.


Better documented, but I haven't found the documentation to always be kept up to date.  This "hello world" tutorial which is likely the first thing anyone's going to find when learning how to embed SM, won't even compile with the current version:
https://developer.mozilla.org/en-US/docs/Mozilla/Projects/SpiderMonkey/How_to_embed_the_JavaScript_engine

I had to do some digging before I found out that JS_NewRuntime and JS_NewContext were merged into one function at some point.  Speed bumps like this are surmountable (and of course MDN is a wiki so I could fix the tutorial myself), but it doesn't exactly inspire confidence.  I found Duktape's documentation to be a lot better when I was first learning to use it.
Title: Re: minisphere 4.2.4
Post by: DaVince on October 24, 2016, 03:02:56 pm
Is there really even a point to switching to anything else? Isn't Duktape serving its purpose just fine on this? Or is this done purely for experimental reasons?
Title: Re: minisphere 4.2.4
Post by: Fat Cerberus on October 24, 2016, 03:39:34 pm
We'll call it "experimental with intent to implement". ;)

As far as Sphere 1.x backward compatibility and migration of existing projects is concerned, Duktape is certainly more than adequate.  However there are a lot of neat new JS constructs from the latest specifications that it doesn't support and probably won't for the foreseeable future, for example:



I have an ongoing dialogue with the developer.  Some work is being done to get some of the easier stuff implemented, but overall, comprehensive ES6+ support is not a near-term goal; Duktape is effectively an ES5 engine + some extras from later specs (like ArrayBuffer).  If I switch to SpiderMonkey or V8 I can get all of this stuff now, plus much better performance thanks to the JIT.
Title: Re: minisphere 4.2.4
Post by: DaVince on October 25, 2016, 10:31:30 am
I see, that makes sense. I guess I need to brush up on my JS too, considering so much stuff is being added.
Title: Re: minisphere 4.2.4
Post by: Eggbertx on October 26, 2016, 10:25:47 pm
Would it be at all possible to use a non-Allegro library to add MP3 support? Some of the old 1.x games had mp3 audio files, and it would be nice to have it for convenience, rather than having to go through, convert each one and then look for where each one is loaded.
Title: Re: minisphere 4.2.4
Post by: Fat Cerberus on October 26, 2016, 11:21:39 pm

Would it be at all possible to use a non-Allegro library to add MP3 support? Some of the old 1.x games had mp3 audio files, and it would be nice to have it for convenience, rather than having to go through, convert each one and then look for where each one is loaded.


Not without relicensing minisphere under the GPL.  There are no MIT/BSD licensed mp3 decoders, sadly. :-\
Title: Re: minisphere 4.2.4
Post by: DaVince on October 27, 2016, 06:30:41 am
Well, at least there's this (https://github.com/audiocogs/mp3.js) to decode MP3 in JS. The code looks pretty browser-independent. GPL2-licensed tho.
Title: Re: minisphere 4.2.4
Post by: Flying Jester on October 27, 2016, 03:51:14 pm
There are also LGPL-licenses MP3 decoders, which would not interfere with the permissive license.
Title: Re: minisphere 4.2.4
Post by: Fat Cerberus on October 28, 2016, 01:04:00 am
I finally caved and the latest minisphere build supports type coercion for boolean arguments of Sphere v1 functions.  It turns out that a lot of legacy Sphere code assumed this and did things like passing 1 and 0 for true and false, which broke several games in minisphere due to a stricter type check which throws a TypeError for non-boolean values.  Sphere v2 APIs still use the stricter check, of course.

I also added native support for Promises (using a polyfill).

edit: And with that change, Trial and Error now runs in minisphere unmolested. ;D  Screenshot attached - also note minisphere's CPU usage--and that's with the debugger attached!
Title: Re: minisphere 4.2.4
Post by: Fat Cerberus on October 29, 2016, 02:00:45 am
In case anyone hasn't figured it out by now, my new goal is to get Kefka's Revenge to run in minisphere 4.3 without modification.  As I've come to learn over the course of development, that game has some of the worst Sphere 1.5 code ever written, relies heavily on type coercion for function parameters, and requires pretty much perfect emulation of the Sphere 1.5 map engine to function properly.  If minisphere can run it unmolested, that speaks volumes for Sphere 1.5 compatibility in general.  Which is great, because that means there'd then be no excuse at all for anyone not to upgrade. ;)

It was a particular challenge to emulate type coercion for map layer parameters.  minisphere supports named map layers (like Sphere 1.6), so any layer index passed as a string will usually be interpreted as a layer name, the lookup will then fail and cause an error.  Tonight I finally came up with a creative solution to that:

https://github.com/fatcerberus/minisphere/blob/master/src/engine/map_engine.c#L1630-L1643

If a string is passed, the engine first looks up the string in the list of layer names.  If that fails, it then uses strtol() to convert the string to an integer and performs an index lookup instead.  Only if THAT fails is the layer reported not found.
Title: Re: minisphere 4.2.4
Post by: Radnen on October 29, 2016, 04:04:03 am
lol at Pokemon Uranium ;)

You're doing an amazing job! It almost makes me want to work on my games again. But I'll save that for sometime early next year or in December. I'm in the process of buying my first house (which I'm sharing with my brother), I'll set up a game room, add my computer and when I find some time, I might just pick up Sphere again.
Title: Re: minisphere 4.2.4
Post by: Fat Cerberus on October 29, 2016, 12:41:44 pm
Sphere 1.5 typing for boolean parameters is odd:
https://github.com/sphere-group/sphere/blob/master/sphere/source/engine/parameters.cpp#L53-L74

Sphere converts numbers, strings, etc. to boolean but will throw if an object is passed (normally truthy).  For legacy support I don't think it's worth it to go out of my way to match those semantics exactly, normal implicit type conversion is more consistent and works fine for compatibility.  Another oddity is that parameters specified as integers get the implicit conversion, while floating point parameters are strict.
Title: Re: minisphere 4.2.4
Post by: Radnen on October 29, 2016, 03:26:19 pm

Sphere converts numbers, strings, etc. to boolean but will throw if an object is passed (normally truthy).  For legacy support I don't think it's worth it to go out of my way to match those semantics exactly...


And since it threw an error, there probably aren't any legacy games that would use objects in place of booleans.
Title: Re: minisphere 4.2.4
Post by: DaVince on October 29, 2016, 07:47:48 pm
Jesus, Kafka's Revenge, huh? Good luck getting that abomination to run perfectly. It had issues even between Sphere versions. :P
Title: Re: minisphere 4.2.4
Post by: Eggbertx on October 29, 2016, 07:57:25 pm
I noticed I had been getting undefined errors (kb) in my script, so I reinstalled everything and noticed the API had changed (or for whatever reason I had been doing something wrong in the first place and had somehow not noticed it). I'm unable to download the installer from the releases page on the Github repo. In Firefox, it just fails and in Chrome it says there's a network error. In order to download it, I need to go into the bash subsystem (Windows 10 insider) and wget it.
Title: Re: minisphere 4.2.4
Post by: Fat Cerberus on October 29, 2016, 08:35:58 pm
Yeah, I renamed 'kb' to 'keyboard' a few versions ago for clarity.  There might some other minor changes too, might be worth flipping through the documentation to double check.

Once minisphere 5.0 is released I plan to keep the API stable so that any code written for earlier versions will continue to run unchanged on later versions.  Not sure how feasible that'll be but I'd like to try at least.
Title: Re: minisphere 4.2.4
Post by: Fat Cerberus on October 29, 2016, 08:41:56 pm

Jesus, Kafka's Revenge, huh? Good luck getting that abomination to run perfectly. It had issues even between Sphere versions. :P


I know, which is what makes it such an interesting challenge. :)

Honestly I got bored, there were no more new features to add to the engine so I said "okay, let's try to make minisphere more compatible with horrible Sphere 1.5 code" :P
Title: Re: minisphere 4.2.4
Post by: Eggbertx on October 30, 2016, 01:28:00 am
As a suggestion, could you integrate the terminal module into the engine itself so it wouldn't be affected by things like loops and normal keyboard input?
Title: Re: minisphere 4.2.4
Post by: Fat Cerberus on October 30, 2016, 02:54:22 am

As a suggestion, could you integrate the terminal module into the engine itself so it wouldn't be affected by things like loops and normal keyboard input?


Hm... that might actually be possible to do.  Actually it would be great if the entire miniRT threader could be more seamlessly integrated so that calling system.run() would update threads automatically.  Something like how update/render scripts work in Sphere 1.x, but available engine-wide.  Thanks for the suggestion, I'll add a note to look into this when I get some time. :D
Title: Re: minisphere 4.2.4
Post by: Fat Cerberus on October 30, 2016, 03:19:20 am
@Eggbert: For the time being when using the terminal module, you have an implicit dependency on miniRT threads, so to enter the event loop you need to call threads.join() (this may need some refactoring if you're not already taking advantage of the threads module - like I said, full integration with the built-in event queue would be nicer).

Refer to miniRT docs for how to use threads.

Thanks again for the feature suggestion, finally someone gives me a challenge! ;D
Title: Re: minisphere 4.2.4
Post by: Eggbertx on October 30, 2016, 02:01:37 pm
Oh, alright. I thought it was handling threads itself already. And luckily I don't have to do any refactoring. My Personlib that I mentioned in another thread already makes good use of them. It wasn't an initial reason for creating the library but I was trying to figure out why updating/rendering was acting weird, and I noticed that I could use threading to my benefit.

Also, I'm not sure if it was like this in the original engine as well, but is it not possible to get a peron's spriteset's filename by doing GetPersonSpriteset(person_name).filename?
Title: Re: minisphere 4.2.4
Post by: Fat Cerberus on October 30, 2016, 10:18:17 pm

Also, I'm not sure if it was like this in the original engine as well, but is it not possible to get a peron's spriteset's filename by doing GetPersonSpriteset(person_name).filename?


I'm not sure, I'd have to verify against Sphere 1.5.  If it's missing, I'll add it.  There are a few legacy objects which support file name tracking and then others that don't, so I'm not 100% positive.  In any case that's a worthwhile feature to make standard for Sphere v2, in light of SphereFS being a thing, and would be easy enough to return a normalized filename since SphereFS already requires normalization per my specification for it:
https://github.com/sphere-group/pegasus/blob/master/design/SphereFS.md
Title: Re: minisphere 4.2.4
Post by: Fat Cerberus on October 30, 2016, 11:46:01 pm
In line with Eggbert's feature request above, I'm working on implementing better async support for Sphere v2.  The current design I have in mind is based on a single function, system.defer(), which works something like setTimeout() in browsers.  Basically this would be the beginning of introducing some sort of event loop binding to Sphere, which will be needed in any case if we ever want to have a WebSphere.
Title: Re: minisphere 4.2.4
Post by: Fat Cerberus on November 01, 2016, 12:37:20 am

Also, I'm not sure if it was like this in the original engine as well, but is it not possible to get a peron's spriteset's filename by doing GetPersonSpriteset(person_name).filename?


So it turns out that the above should have worked, but clone_spriteset had a bug in it where it didn't copy the filename.  I'll fix it.

Little known fact: GetPersonSpriteset() returns a copy of the spriteset object and not the original one assigned to the entity.  One of the many oddities in Sphere 1.x I discovered through reverse engineering while reimplementing the map engine.  To save memory and avoid killing performance, minisphere only clones the spriteset structures and not the underlying textures--which works because Images in Sphere are read-only.
Title: Re: minisphere 4.2.4
Post by: Fat Cerberus on November 02, 2016, 12:24:25 pm

lol at Pokemon Uranium ;)


Hehe.  It only disappoints me that it wasn't made in Sphere. :'(
Title: Re: minisphere 4.2.4
Post by: DaVince on November 04, 2016, 08:53:14 am
So I'm on Windows right now and ran the installer, and... minisphere just keeps crashing every time I try to run it, no matter what code I feed it. This is on 64-bit Windows 7, any way to debug it?

Speaking of debugging, when I tried pressing "Debug", minisphere *did* run once, but it kept waiting on SSJ and Sphere Studio flat-out told me it failed to start up the debugger.
Title: Re: minisphere 4.2.4
Post by: Fat Cerberus on November 04, 2016, 08:59:02 am
Interesting, I'd need to set up a Win7 VM to test, it works fine for me on Win10.  As for Sphere Studio, yeah, it'll time out after about 10 seconds of not being able to connect.

Although... I think I might have fixed a crash or two since 4.2.4 was released?  I'll look into it and expedite the 4.3 launch if that's the case.
Title: Re: minisphere 4.2.4
Post by: DaVince on November 04, 2016, 09:28:19 am
That would be nice, hopefully that would help. I intend on writing a game for someone who runs Vista, so it's rather important that the engine works for them too. :P
Title: Re: minisphere 4.2.4
Post by: Fat Cerberus on November 04, 2016, 10:48:41 am
minisphere now differentiates between command-line debug sessions (SSJ) vs. Sphere Studio interactive (SSJ Blue).  Screenshots attached.
Title: Re: minisphere 4.2.4
Post by: Fat Cerberus on November 04, 2016, 01:28:53 pm

So I'm on Windows right now and ran the installer, and... minisphere just keeps crashing every time I try to run it, no matter what code I feed it. This is on 64-bit Windows 7, any way to debug it?


Here's the console log from my test on a Win7 VM:
Code: [Select]

C:\src\spectacles-i>spherun --verbose 4 dist
minisphere 4.2.4 JS game engine (x64)
a lightweight JavaScript-powered game engine
(c) 2015-2016 Fat Cerberus

parsing command line
    game path: C:/src/spectacles-i/dist/
    fullscreen: off
    frameskip limit: 5 frames
    sleep when idle: yes
    console verbosity: V4
    debugger mode: passive

initializing Allegro 5.2.1.1
initializing Dyad 0.2.0
initializing Duktape 1.99.99 (v1.5.0-687-g4cad05a)
loading system configuration
opening kevfile #0 as `#/system.ini`
initializing async subsystem
initializing Galileo subsystem
initializing audio subsystem
initializing input subsystem
opening kevfile #1 as `#/minisphere.conf`
    `#/minisphere.conf` doesn't exist
reading key `keymap_Player1_MENU` from kevfile #1
reading key `keymap_Player1_UP` from kevfile #1
reading key `keymap_Player1_DOWN` from kevfile #1
reading key `keymap_Player1_LEFT` from kevfile #1
reading key `keymap_Player1_RIGHT` from kevfile #1
reading key `keymap_Player1_A` from kevfile #1
reading key `keymap_Player1_B` from kevfile #1
reading key `keymap_Player1_X` from kevfile #1
reading key `keymap_Player1_Y` from kevfile #1
reading key `keymap_Player2_MENU` from kevfile #1
reading key `keymap_Player2_UP` from kevfile #1
reading key `keymap_Player2_DOWN` from kevfile #1
reading key `keymap_Player2_LEFT` from kevfile #1
reading key `keymap_Player2_RIGHT` from kevfile #1
reading key `keymap_Player2_A` from kevfile #1
reading key `keymap_Player2_B` from kevfile #1
reading key `keymap_Player2_X` from kevfile #1
reading key `keymap_Player2_Y` from kevfile #1
reading key `keymap_Player3_MENU` from kevfile #1
reading key `keymap_Player3_UP` from kevfile #1
reading key `keymap_Player3_DOWN` from kevfile #1
reading key `keymap_Player3_LEFT` from kevfile #1
reading key `keymap_Player3_RIGHT` from kevfile #1
reading key `keymap_Player3_A` from kevfile #1
reading key `keymap_Player3_B` from kevfile #1
reading key `keymap_Player3_X` from kevfile #1
reading key `keymap_Player3_Y` from kevfile #1
reading key `keymap_Player4_MENU` from kevfile #1
reading key `keymap_Player4_UP` from kevfile #1
reading key `keymap_Player4_DOWN` from kevfile #1
reading key `keymap_Player4_LEFT` from kevfile #1
reading key `keymap_Player4_RIGHT` from kevfile #1
reading key `keymap_Player4_A` from kevfile #1
reading key `keymap_Player4_B` from kevfile #1
reading key `keymap_Player4_X` from kevfile #1
reading key `keymap_Player4_Y` from kevfile #1
disposing kevfile #1 no longer in use
initializing sockets subsystem
    Dyad.c 0.2.0
initializing spriteset manager
initializing map engine
initializing persons manager
creating new mixer #0 at 44 kHz
    format: 2ch 44100 Hz, 16-bit
initializing JS script manager
setting up jump points for longjmp
searching for a game to launch
opening `C:/src/spectacles-i/dist/` in sandbox #0
opening SPK #0 as `C:/src/spectacles-i/dist/`
failed to open SPK #0
parsing JSON manifest for sandbox #0
    title: Spectacles: Bruce's Story
    author: Author Unknown
    resolution: 320x200
initializing Sphere v1 API (v2.0)
creating new mixer #1 at 44 kHz
    format: 2ch 44100 Hz, 16-bit
reading key `Arrow` from kevfile #0
loading image #0 as `#/pointer.png`
reading key `UpArrow` from kevfile #0
loading image #1 as `#/up_arrow.png`
reading key `DownArrow` from kevfile #0
loading image #2 as `#/down_arrow.png`
reading key `ssWindowStyle` from kevfile #0
reading 16x16 image #3 from open file
reading 16x16 image #4 from open file
reading 16x16 image #5 from open file
reading 16x16 image #6 from open file
reading 16x16 image #7 from open file
reading 16x16 image #8 from open file
reading 16x16 image #9 from open file
reading 16x16 image #10 from open file
reading 16x16 image #11 from open file
initializing Sphere v2 L0 API
    sphere_glsl_shader_support
    sphere_stateful_rng
    sphere_v1_compatible_api
    minisphere_ssj_api
creating new mixer #2 at 44 kHz
    format: 2ch 44100 Hz, 16-bit
loading image #12 as `icon.png`
initializing render context at 320x200
    shader support: yes
cloning image #13 from source image #12


So it appears the crash is in clone_image().  Unfortunately that doesn't really help me much.  I'll need to get Visual Studio 2015 running on this VM so I can debug it properly.

@DaVince: Just for the heck of it though... could you try --verbose 4 locally and see if it crashes in the same spot?

edit: Hm, the crash appears to be Allegro-related.  Invariably regardless of where it crashes the top of the stack looks like this:
Code: [Select]

0000000000000000() Unknown
> spherun.exe!_al_opengl_set_blender() Unknown


It tries to call a function from a null pointer, crashing the process.  I'll report it to the Allegro devs and see if I can gain any more insight.
Title: Re: minisphere 4.2.4
Post by: DaVince on November 04, 2016, 03:32:03 pm
Quote
@DaVince: Just for the heck of it though... could you try --verbose 4 locally and see if it crashes in the same spot?

Um, how do I do that? What folder do I run minisphere in? The folder where the ssproj file is? Because it refuses to recognize the ssproj or the folder in general if I try to run it from the console...

Edit: oh, got it. Need to run it from the dist folder.
Edit 2: it crashes before even outputting anything at all, unfortunately. Maybe I'm doing this wrong? minisphere . --verbose 4
(Edit 3: I'm off right now btw, will respond again tomorrow)
Title: Re: minisphere 4.2.4
Post by: Fat Cerberus on November 04, 2016, 03:42:43 pm

Quote
@DaVince: Just for the heck of it though... could you try --verbose 4 locally and see if it crashes in the same spot?

Edit 2: it crashes before even outputting anything at all, unfortunately. Maybe I'm doing this wrong? minisphere . --verbose 4


You need to use the spherun command instead of minisphere.  Windows is odd in that programs without a console window won't output anything at all even if you run them from the terminal.
Title: Re: minisphere 4.2.4
Post by: DaVince on November 05, 2016, 06:39:38 am


Quote
@DaVince: Just for the heck of it though... could you try --verbose 4 locally and see if it crashes in the same spot?

Edit 2: it crashes before even outputting anything at all, unfortunately. Maybe I'm doing this wrong? minisphere . --verbose 4


You need to use the spherun command instead of minisphere.  Windows is odd in that programs without a console window won't output anything at all even if you run them from the terminal.

Oh, I know about that, that's why I always run them in a terminal. But it still wasn't outputting anything by using the minisphere command.

Anyway, after running spherun instead, my output is different. The last part of the output looks like this:

Code: [Select]
...
creating image #270 as 8x12 subimage of image #15
creating image #271 as 8x12 subimage of image #15
listening for debugger on TCP port 1208
opening socket #0 to listen on 1208
    backlog: up to 1
initializing JS module `main.js`


minisphere's "loading..." text pops up and that's where it locks up.
Title: Re: minisphere 4.2.4
Post by: Fat Cerberus on November 05, 2016, 11:54:18 am

Oh, I know about that, that's why I always run them in a terminal. But it still wasn't outputting anything by using the minisphere command.


No, that's what I meant - even if you have a terminal window, if the executable was compiled as a GUI application, Windows won't allow it to print anything.

Anyway, that log is interesting - it's apparently gotten far enough to create 200+ images (so not the OpenGL issue I thought it was), but then dies trying to load the main module, just after firing up the debug server.

What happens if you try to run the game with SSJ?  From the dist folder, do: ssj -r .
Title: Re: minisphere 4.2.4
Post by: DaVince on November 05, 2016, 12:18:05 pm
I get the following. I'm thinking the weird garbage characters are just some encoding error?

Code: [Select]
SSJ 4.2.4 Sphere JavaScript debugger (x64)
a powerful JavaScript debugger for minisphere
(c) 2015-2016 Fat Cerberus

starting C:/Users/VB/Dropbox/projects/sphere/2016/TEST/dist/... OK.
connecting to 127.0.0.1:1208... OK.
verifying... ←[31;1merror!←[m
Title: Re: minisphere 4.2.4
Post by: Fat Cerberus on November 05, 2016, 12:24:22 pm
The garbage characters are ANSI escape codes, it's supposed to print that "error" in red but I guess the Win7 console doesn't support it.  I assume in this case the engine crashed completely and not just locked up?  If it did lock up, then the error is interesting because it suggests the SSJ handshake failed (if it crashed then the error is not a surprise).

I'll look into this further on my Windows 7 VM.  I have to get around the OpenGL driver issue first though.

Does it crash even if you give it a Sphere 1.5 game to run?
Title: Re: minisphere 4.2.4
Post by: DaVince on November 05, 2016, 12:33:39 pm
It just crashed in the same way it always does, so I guess this is nothing unusual. I just noticed it runs Sphere 1.5 games entirely fine though... Should I send you the project I have?
Title: Re: minisphere 4.2.4
Post by: Fat Cerberus on November 05, 2016, 12:34:21 pm
Yes, please.  It might be a bug in the Sphere v2 initialization code.
Title: Re: minisphere 4.2.4
Post by: DaVince on November 05, 2016, 12:38:40 pm
Okay; it's attached. Note that I hadn't even started coding _anything_ yet, so you're probably right about the initialization thing.

Note that I'm also running this game from a different folder than the default one (Dropbox\projects\Sphere\2016\ to be exact). I dunno if that would influence anything.
Title: Re: minisphere 4.2.4
Post by: Fat Cerberus on November 05, 2016, 01:08:45 pm
Thanks, it crashes on Win10 too.  And, well, this is embarassing.  It crashes here:

Code: (c) [Select]

source = sfs_fslurp(g_fs, filename, NULL, &source_size);
code_string = lstr_from_cp1252(source, source_size);
free(source);


source is NULL because main.js didn't make it into the dist (it's not mentioned in the Cellscript).  This ultimately leads to a crash.

It should have given you an error that main.js didn't exist rather than crashing, I'll need to fix that.

edit: By the way, you don't need a game() function for Sphere v2, and in fact unless the engine is running in 1.x compatibility mode the function will be completely ignored.  Sphere v2 should be written like you'd write a Node.js script.
Title: Re: minisphere 4.2.4
Post by: DaVince on November 05, 2016, 01:12:40 pm
Well, now I also wonder: how did main.js not get referenced? I literally just created a new project in Sphere Studio and those files were already included. I guess there's something to fix there too?
Title: Re: minisphere 4.2.4
Post by: Fat Cerberus on November 05, 2016, 01:18:54 pm
Yeah, the project template is probably broken.  I always forget to update that when I reorganize stuff.  It was originally made for Sphere 1.x where all scripts go into the "scripts" folder.  For now you can fix it by adding this to the Cellscript:

Code: [Select]

install(files("main.js"));


Sorry about that.  :-[
Title: Re: minisphere 4.2.4
Post by: DaVince on November 05, 2016, 01:32:52 pm
Thanks, I'll do that. That seems to work, at least! If I keep my other scripts in scripts/, should I be fine for the time being?

Edit: wait, it doesn't work properly after all. My script has one GetKey() but it is ignored and minisphere just quits right away. Same deal if I do while (!IsAnyKeyPressed());...
Edit 2: Oh wait. I guess game() isn't where Sphere starts by default anymore. Gotta start reading up on the documentation!
Title: Re: minisphere 4.2.4
Post by: Fat Cerberus on November 05, 2016, 01:57:05 pm

Oh wait. I guess game() isn't where Sphere starts by default anymore. Gotta start reading up on the documentation!


Yep, I mentioned that above, guess you missed it.  minisphere completely ignores the game() function if you're running in Sphere v2 mode because everything is a module (like Node).  Write your scripts like you were writing a Node.js program.  Obviously it's not ignored in Sphere 1.x compatibility mode because that would break... well, pretty much everything. :P
Title: Re: minisphere 4.2.4
Post by: DaVince on November 05, 2016, 01:59:14 pm
Oh, I did miss your edit. I'll be doing it like that from now on then. :)
Title: Re: minisphere 4.2.4
Post by: Fat Cerberus on November 05, 2016, 02:29:57 pm
Okay, both the crash and project template are fixed in the latest build on GitHub.  Thanks for bearing with me on this.

Unfortunately Sphere v2 mode doesn't get much testing; I still need to run Specs in v1 mode because it relies heavily on RequireScript() and stuff going into the global scope, which doesn't work when everything runs as a module.  Some example projects I can test against would remedy that, but I've been too lazy to put anything together.
Title: Re: minisphere 4.2.4
Post by: DaVince on November 05, 2016, 02:41:54 pm
Nice, will get it right away. :)

And I'm fully planning on creating a Sphere game with the new API in the coming weeks, so hopefully that will help. By the way, the API seems outdated again; Keyboard is undefined. Um, gonna check that changelog again.
Edit: I see, the API reference is actually ahead this time and it's still "keyboard" in this version. Gotcha.
Title: Re: minisphere 4.2.4
Post by: Fat Cerberus on November 05, 2016, 02:53:54 pm
Yeah, I'll release 4.3.0 in a day or two and then everything will be in sync again. :)
Title: Re: minisphere 4.2.4
Post by: Fat Cerberus on November 05, 2016, 03:02:33 pm
One quick change I want to make before release is for `Dispatch.later()` to be frame-perfect rather than wall clock timed.  This way it can be used to accurately time in-game events.  I might still have it take a value in seconds, but the actual timing would be based on frames under the hood.  For example if running at 60fps, then 2 seconds translates to "120 frames".
Title: Re: minisphere 4.2.4
Post by: Flying Jester on November 05, 2016, 03:39:44 pm


Oh, I know about that, that's why I always run them in a terminal. But it still wasn't outputting anything by using the minisphere command.


No, that's what I meant - even if you have a terminal window, if the executable was compiled as a GUI application, Windows won't allow it to print anything.


It will if you manually fflush after every newline.
Title: Re: minisphere 4.3.0
Post by: Fat Cerberus on November 07, 2016, 11:52:40 am
minisphere 4.3 is out, and it includes a ton of new stuff: the new Dispatch API, promises, Reflect, miniRT enhancements, frame-perfect timing, better Sphere 1.x compatibility (it now runs Kefka's Revenge!), API refactoring, and a bunch of bug fixes.  Get it today--and get eaten by a giant pig tomorrow! :pig_nose:

I'm especially proud of the Dispatch API, as it brings the functionality of Sphere 1.x update, render and delay scripts engine-wide.  And you can set up multiple update and render jobs, so it ALSO works like the old hook-list script with the added bonus of being able to control the order using priorities.  It's quite useful!
Title: Re: minisphere 4.3.0
Post by: Fat Cerberus on November 07, 2016, 12:36:34 pm
@Eggbert: It looks like you forgot to update the .tar.gz for the Arch release again, it's still the one for 4.2.4.
Title: Re: minisphere 4.3.0
Post by: Eggbertx on November 07, 2016, 12:37:32 pm
I updated the AUR package.
Title: Re: minisphere 4.3.0
Post by: DaVince on November 07, 2016, 03:20:16 pm

minisphere 4.3 is out, and it includes a ton of new stuff: the new Dispatch API, promises, Reflect, miniRT enhancements, frame-perfect timing, better Sphere 1.x compatibility (it now runs Kefka's Revenge!), API refactoring, and a bunch of bug fixes.  Get it today--and get eaten by a giant pig tomorrow! :pig_nose:

I'm especially proud of the Dispatch API, as it brings the functionality of Sphere 1.x update, render and delay scripts engine-wide.  And you can set up multiple update and render jobs, so it ALSO works like the old hook-list script with the added bonus of being able to control the order using priorities.  It's quite useful!

Nice! I've been wanting to try out the new features. :)
Title: Re: minisphere 4.3.1
Post by: Fat Cerberus on November 08, 2016, 10:53:14 am
Quick update: minisphere 4.3.1 adds an engine-provided frame loop that runs as long as there are active Dispatch jobs.  While seemingly a small change, this represents a fairly large paradigm shift.  Instead of this:

Code: (javascript) [Select]

while (system.run()) {
    // render stuff here
    screen.flip();
    // update stuff here
}


The idiomatic Sphere v2 way to write a game is now:
Code: (javascript) [Select]

Dispatch.onUpdate(doUpdate);
Dispatch.onRender(doRender);

function doUpdate()
{
    // update stuff here
}

function doRender()
{
    // render stuff here
}


It's a bit more complicated, but effectively paves the way for cross-compatibility with Oozaru.  And if you're only targeting minisphere, then you can still use the classic method. :)
Title: Re: minisphere 4.3.1
Post by: DaVince on November 08, 2016, 11:07:04 am
Would the Exit() and Abort() functions cancel all jobs currently dispatched? Or would I have to cancel each job through Dispatch.cancel(job)? Is there a way to cancel all jobs (like Dispatch.cancelAllJobs()), and would that exit the engine right away or would you still have a brief moment to dispatch a whole new job right after that?
Title: Re: minisphere 4.3.1
Post by: Fat Cerberus on November 08, 2016, 11:13:00 am
The engine won't quit until you return from the main script, same as Sphere 1.x.  It's just that if there ARE active jobs at that time (including one-time jobs), it will run a gameloop until they have all finished running.

There's nothing to cancel all jobs right now, you'd have to cancel them individually.  That said, system.exit() and system.abort() (and indeed, as you point out, the Sphere v1 equivalents) still work as before and close the engine immediately.
Title: Re: minisphere 4.3.1
Post by: Fat Cerberus on November 08, 2016, 02:27:04 pm
Damn, I wish I had caught this this morning.  There's a minor glitch in the current release: Dispatch jobs will continue to be processed even after an uncaught exception, possibly rendering over the error screen (I'm not really sure what would happen if another error is thrown at this point, probably a crash).  All jobs should have been canceled in that case, but due to the oversight they aren't.  I fixed it already in the latest build, but I don't think it's pressing enough to warrant a hotfix.

edit: Hm, it seems the engine will crash if an error is thrown in an update or render call dispatched from the main event loop.  So that's something else to fix, but also not a big deal (if you don't catch an error you were going to terminate anyway, it's just that normally that wouldn't cause an outright crash).  It looks like I'll need to go through and test a few different scenarios to make sure all the edge cases are covered.  Expect a 4.3.2 release in a few days with Dispatch API fixes.  For the time being everything is completely usable, there just might be some landmines involved. :P
Title: Re: minisphere 4.3.2
Post by: Fat Cerberus on November 09, 2016, 12:59:31 pm
Just released minisphere 4.3.2 with a bunch of fixes.  The crashes mentioned above are fixed, and the new project template has been greatly improved.  Notably, this version also improves priority handling for Dispatch.onUpdate(): Jobs with higher priority are now executed first.  Renders are still processed low->high of course.
Title: Re: minisphere 4.3.2
Post by: Eggbertx on November 10, 2016, 03:30:08 pm
Something that I didn't notice until now for some reason, in Sphere 1.x, if Abort() was called (whether via script error or called from a script) and the game had been loaded from a startup game, Sphere went back to the startup game, but this isn't done in minisphere. Also, if I press Alt-Enter from the startup game to toggle window view (when not set in the command line), when the game loads it goes back to fullscreen.
Title: Re: minisphere 4.3.2
Post by: Fat Cerberus on November 10, 2016, 03:51:18 pm
Thanks for the heads-up on Abort(), the bug is here:
https://github.com/fatcerberus/minisphere/blob/master/src/engine/main.c#L768

I call exit() after displaying the error screen.  It should call `exit_game(false)` instead, which it would if I just remove the exit().  Not sure why I put that there, I'll fix it for 4.3.3.

I'll look into the full screen issue later too, that sounds like a bug since it's not the intended behavior.

Tip: Abort() doesn't actually abort unconditionally but rather throws a catchable exception (Sphere games historically didn't use try/catch that much).  If you want an unconditional error, call Sphere v2 system.abort() instead.
Title: Re: minisphere 4.3.2
Post by: Eggbertx on November 10, 2016, 04:02:31 pm
Oh, alright. Also, why was fs changed to FS?
Title: Re: minisphere 4.3.2
Post by: Fat Cerberus on November 10, 2016, 10:27:35 pm

Oh, alright. Also, why was fs changed to FS?


The release notes for 4.3 explain this.  Mostly it was done to match other JS namespaces like Math and Reflect.  It's not really an object representing the file system so much as "the place where file system functions live."  So it gets an uppercase name.  A concrete global "this is the file system" variable only really makes sense if the engine supports multiple independent file systems, which doesn't seem like something we need in Sphere.

screen on the other hand was left lowercase because it's an actual Surface object, just one that happens to represent the backbuffer.  It's not a namespace but a true object in the OOP sense.  The alternative is to call it Surface.Screen for consistency with Mixer.Default etc. but that's a bit verbose for an object that will be referenced all the time. ;)

tl;dr: API consistency concerns.
Title: Re: minisphere 4.3.2
Post by: Fat Cerberus on November 11, 2016, 02:23:55 am
Hm, it turns out I was wrong about Abort()'s behavior: In Sphere 1.5 it actually is an unconditional error; wrapping it in a try/catch has no effect.  minisphere currently implements it as a throw which is incompatible, so I'll have to fix that.
Title: Re: minisphere 4.3.2
Post by: Fat Cerberus on November 11, 2016, 03:15:22 am
@Eggbert: The two bugs you found, along with the incorrect Abort() throwing behavior, are fixed in git and will be in the 4.3.3 release.  I'll hold off on that release for a bit longer in case you find more glitches. ;)
Title: Re: minisphere 4.3.2
Post by: Eggbertx on November 11, 2016, 03:50:01 am


Oh, alright. Also, why was fs changed to FS?


The release notes for 4.3 explain this.  Mostly it was done to match other JS namespaces like Math and Reflect.  It's not really an object representing the file system so much as "the place where file system functions live."  So it gets an uppercase name.  A concrete global "this is the file system" variable only really makes sense if the engine supports multiple independent file systems, which doesn't seem like something we need in Sphere.

screen on the other hand was left lowercase because it's an actual Surface object, just one that happens to represent the backbuffer.  It's not a namespace but a true object in the OOP sense.  The alternative is to call it Surface.Screen for consistency with Mixer.Default etc. but that's a bit verbose for an object that will be referenced all the time. ;)

tl;dr: API consistency concerns.

What about system then?
Title: Re: minisphere 4.3.2
Post by: Fat Cerberus on November 11, 2016, 03:54:49 am
It has properties (engine name, version, etc.), so I treat it as a real object representing the engine.  In fact it originally was called "engine" a while ago, don't remember why I changed it.  It is kind of a gray area though, unlike screen.
Title: Re: minisphere 4.3.2
Post by: Fat Cerberus on November 11, 2016, 01:54:44 pm
I decided to release minisphere 4.3.3 now since it adds a few useful features: Object.assign(), and a new "test" module based on http://wiki.commonjs.org/wiki/Unit_Testing/1.0

The test module is used for automating unit tests.  You call test.run() with object full of functions called "testWhatever" and it runs them all for you:
Code: (javascript) [Select]
const assert = require('assert');
const test   = require('test');

test.run({
    testFatburner: function() {
        var scott = existence.find("Scott Starcross");
        var maggie = existence.find("maggie");
        var fatburner = existence.find("Holy Blade Fatburner");
        scott.equipWeapon(fatburner);
        scott.attack(maggie);
        assert.ok(maggie.deceased);
    },

    testMunch: function() {
        var scott = existence.find("Scott Starcross");
        var maggie = existence.find("maggie");
        maggie.munch(scott);
        assert.ok(!existence.has(scott));
    }
});
Title: Re: minisphere 4.3.3
Post by: Radnen on November 12, 2016, 01:03:55 am
Ah, that explains the Link query from the other post. ;) Nice work!
Title: Re: minisphere 4.3.3
Post by: Fat Cerberus on November 12, 2016, 11:43:19 am
The next version of minisphere will rename link() to link.Q() to make it more obvious when you're dealing with a query.

Old way (minisphere 4.3.3):
Code: (javascript) [Select]

const link = require('link');

link(array).where(...).each(function(v) {
    console.log(v);
});


New:
Code: (javascript) [Select]

const link = require('link');

link.Q(array).where(...).forEach(function(v) {
    console.log(v);
});
Title: Re: minisphere 4.3.3
Post by: Eggbertx on November 13, 2016, 05:07:50 pm
It looks like Color.of isn't defined.
Title: Re: minisphere 4.3.3
Post by: Fat Cerberus on November 13, 2016, 07:51:01 pm

It looks like Color.of isn't defined.


Looks like you're right, I'll fix that for the next update.  The function exists in the engine but I forgot to register it with Duktape it on startup.
Title: Re: minisphere 4.3.3
Post by: Fat Cerberus on November 16, 2016, 02:55:27 am
Quick progress update: The next version of minisphere will be v4.4.0.  It gets another minor version bump due to Link being replaced in the standard library by the new "from" module which provides most of the same basic functionality.

Link is still more feature-rich, but in the interest of future standardization for the Sphere v2 API I thought it would be better to include something a bit more simple out of the box.  Link has a lot of experimental stuff that, while it's really useful, I don't want to accidentally codify as an official part of the Sphere API.  That kind of thing is how JavaScript ended up with all kinds of weird semantics because to fix them now would break websites :P

@Radnen: Thanks for making Query.js and Link all those years ago.  Having some fully documented LINQ-style query functionality available out of the box will be a really great thing for Sphere, I think.  It's already proven its worth for me in the Spectacles battle engine! :D
Title: Re: minisphere 4.3.3
Post by: Radnen on November 16, 2016, 09:50:07 pm
Yeah, I agree with you there, a "basic" version of link would be great to package with minisphere, and those that need the horsepower can upgrade to Link; In short, I endorse.
Title: Re: minisphere 4.3.4
Post by: Fat Cerberus on November 17, 2016, 09:59:08 am
Okay, new version 4.3.4 removes Link and replaces it with the new from module (I didn't bump to 4.4 after all, it barely counts as a new feature):

Code: (javascript) [Select]
const from = require('from');

var a1 = [ 1, 2, 3, 4, 5, 6, 7, 8, 9, 10 ];
var a2 = from.Array(a1)
    .where(function(n) { return n % 2 == 0 })
    .select(function(n) { return n + " pigs"; });
console.log(a2);  // 2 pigs,4 pigs,6 pigs,8 pigs,10 pigs


All query methods are fully documented in docs/miniRT-api.txt.

I also fixed Color.of() which was missing in previous releases, originally reported by Eggbert.
Title: Re: minisphere 4.3.4
Post by: Fat Cerberus on November 19, 2016, 04:42:07 pm
The next big project I'd like to start on is implementing full TypeScript support.  I removed the experimental support for it in minisphere 3.0, because A) it was done on demand by the engine, leading to painful loading times for large games, and B) it didn't support strict typing, entirely defeating the purpose.  Without static typing TS is basically just ES6 + some extra syntax.

So the new plan is to either integrate TS compilation into Cell, or still have the engine do it but then cache the result for later use.  The former seems like a better solution to me, as it'll give more control over the transpilation process.  If the engine does it then you're at the mercy of whatever default options it happens to use.
Title: Re: minisphere 4.3.6
Post by: Fat Cerberus on November 23, 2016, 12:58:45 am
minisphere 4.3.6 is up, now with support for partial application of from() queries:

Code: (javascript) [Select]

// if you do this once:
var itemQuery = from(allItems)
    .where(function(v) { return v.owner === "Scott"; })
    .where(function(v) { return v.type === "healing"; });

// you can then do this as many times as you want:
var toShow = itemQuery
    .skip(itemsPerPage * pageNum)
    .take(itemsPerPage)
    .select();

Title: Re: minisphere 4.3.6
Post by: Fat Cerberus on November 26, 2016, 03:01:13 pm
I was recently looking at the source code for the .NET Framework LINQ methods, and the way it works is interesting.  It essentially does exactly what Link does, but in reverse: Rather than pushing items through a chain of points, it instead pulls them through a chain of iterators.  This is what allows things like OrderBy to be done in-place, vs. Link where they have to run the query.  Operations don't always have to stay in lockstep because everything is regulated through the iterator Next() method.  Thus even non-lazy operations like sorting can appear lazy to the rest of the query.

So naturally, I rewrote the "from" module to use this approach.  So look forward to query improvements in the next minisphere version. ;)
Title: Re: minisphere 4.3.7
Post by: Fat Cerberus on December 02, 2016, 02:07:54 am
minisphere 4.3.7 is out, now with support for Symbol which allows creating semi-private guaranteed-unique property names.  That's useful functionality to have as sometimes closures are too limiting.

I also made vast improvements to the from module.  It's now possible to do queries like this:
Code: (javascript) [Select]

var combos = from(this.combos)
    .where(function(v) { return currentPhase >= v.phase; }.bind(this))
    .random(targets.length)  // partially lazy
    .descending(function(v) { return v.rating; })  // ditto
    .select();


Or this:
Code: (javascript) [Select]

var ignoreList = from(followers)
    .mapTo(function(v) { return v.name; })
    .including([ 'robert' ])  // supplemental items
    .select();
SetPersonIgnoreList(info.name, ignoreList);


Or even THIS (totally contrived but it gets the point across):
Code: (javascript) [Select]

from(people)
    .from(function(v) { return v.pets; })
    .where(function(pet) { return pet.age >= 8.12; })
    .each(function(pet)
{
    pig.devour(pet);
    console.log(pet.name + " got eaten by the pig!");
}
Title: Re: minisphere 4.3.8
Post by: Fat Cerberus on December 04, 2016, 01:25:20 am
from#anyIn() was busted in the last release (it always returned false).  4.3.8 fixes it.
Title: Re: minisphere 4.3.8
Post by: DaVince on December 05, 2016, 05:08:00 am
Nice work with all of this so far!

By the way, I've found it difficult to get started with minisphere while the API keeps changing. Something that worked fine in the previous version is now broken. I guess that's just a risk that comes with chasing a moving target, though.

I'm also not really sure how to start creating your project as if it's a Node.js module - I have really no experience with this. There is, in all honestly, a LOT of new stuff I have to get used to by now, including just getting a picture shown on screen, which I needed to use the prim module for because I just couldn't figure it out otherwise. (I know I'm not a very good coder but still. :P)

The same goes for the Linq-style stuff - it looks really great but usually I don't *quite* understand what half of the selector code does in the examples I see fly by. Comments next to each line, please? Although maybe I just need to read the documentation for that.

Is there a possibility you could focus a little more on creating some simple snippets or tiny example projects on how to use certain parts of the API? That would be good wiki/tutorial material, too. :)


Edit: regarding Windows 7 testing: I'm using a laptop from work nowadays that has Windows 8.1, so I'll be checking that out soon on the notebook that also has Windows 7 on it.
Title: Re: minisphere 4.3.8
Post by: Fat Cerberus on December 05, 2016, 10:11:21 am

Nice work with all of this so far!

By the way, I've found it difficult to get started with minisphere while the API keeps changing. Something that worked fine in the previous version is now broken. I guess that's just a risk that comes with chasing a moving target, though.


Yeah, Sphere v2 is still brand new and comprises a HUGE API surface (built-in engine functions + miniRT, it's a much more comprehensive API than Sphere 1.5's) so I used the 4.x cycle as kind of an experimental testing ground to get all the kinks ironed out (I thought I said something to that effect when I released 4.0.0, maybe not though...).  The goal for minisphere 5.0 is to stabilize the API so that it won't change anymore, at least not in major, fundamental ways like it has throughout the 4.x cycle.  But before that could happen I had to make sure it was presentable - it's important that our frontend be as easy to understand for someone reading the code as possible.  That said, the plan is to release 5.0.0 on the second anniversary of minisphere 1.0 (March 28), so 4.3 will remain current for quite a while.  And if it makes you feel any better: If you take the time to learn the API now, the rug won't be pulled out from under you again, I don't intend to change much in minisphere 5.0. :)

Quote

I'm also not really sure how to start creating your project as if it's a Node.js module - I have really no experience with this. There is, in all honestly, a LOT of new stuff I have to get used to by now, including just getting a picture shown on screen, which I needed to use the prim module for because I just couldn't figure it out otherwise. (I know I'm not a very good coder but still. :P)


Actually don't feel bad about that - that's exactly what the prim module is there for. ;)  miniRT is a first-class part of the Sphere v2 API, so it's not going anywhere.  Galileo is there if you need it for advanced stuff - but if you're just getting started or making simple games, prim is more than sufficient.  The built-in APIs in Sphere v2 are low-level because they are meant to be a foundation, so that you can build any kind of framework on top of them you need.  Games generally shouldn't use them directly; miniRT is there for small-medium sized projects, but if a large project needs something more robust it can build its own custom library on top of the low-level built-in functions.

Also, I feel it's relevant to mention here, the entire Sphere 1.5 API is still there if you need to fall back on it - and it's fully interchangeable with the Sphere v2 stuff.  That was a very important consideration over the course of Sphere v2 development - the two APIs had to be compatible with each other to make migration painless.

Quote

The same goes for the Linq-style stuff - it looks really great but usually I don't *quite* understand what half of the selector code does in the examples I see fly by. Comments next to each line, please? Although maybe I just need to read the documentation for that.


Yeah, I'm not sure any example can really convey its power.  The basic idea is that from(array) converts the array to a query, then each query operator you tack onto the end adds a new link to the "chain".  When the query runs, each item goes through the links in the chain, being transformed or filtered along the way (this is why a lot of the methods take functions as parameters, the examples would be much easier to read if Duktape supported fat arrow functions, but it doesn't).  Whatever makes it to the end without being filtered out is what you get as output.  Here's a query that gives you an array of all even numbers between 1 and 10, as strings:

Code: (javascript) [Select]

var evenStrings = from([ 1, 2, 3, 4, 5, 6, 7, 8, 9, 10 ])
    .where(function(v) { return v % 2 == 0; })  // "where value is even"
    .select(function(v) { return v.toString(); })  // "select value, converted to string"


The best advice I have is to read the documentation, since like I said the current version will be current for a while and the API is not likely to change much for 5.0 anyway.  These are permalinked to the 4.3.8 tag so the links shouldn't break in the future:
https://github.com/fatcerberus/minisphere/blob/v4.3.8/docs/sphere2-api.txt
https://github.com/fatcerberus/minisphere/blob/v4.3.8/docs/miniRT-api.txt

Despite the lack of examples, I did make sure the API documentation always stayed up to date, and it goes into detail where necessary (unlike the Sphere 1.x documentation which gets pretty vague at times...).

Quote

Is there a possibility you could focus a little more on creating some simple snippets or tiny example projects on how to use certain parts of the API? That would be good wiki/tutorial material, too. :)


But yeah, I'll try to do this for minisphere 5.0.
Title: Re: minisphere 4.3.8
Post by: DaVince on December 05, 2016, 02:09:14 pm
Wow, thanks for the very comprehensive answer. I'm in the busy of a very busy lifestyle so I don't know when I can get around to it, but it's good to know I can basically start learning the stuff at any time without huge changes at this point. :)
Title: Re: minisphere 4.3.8
Post by: Fat Cerberus on December 09, 2016, 02:44:29 am
New command-line tool planned for next big release: majin, named after Majin Buu from (what else? :))) Dragon Ball Z.

Majin will let you set up a game manifest from the command line without having to bother with a Cellscript.  For example:

Code: [Select]

mkdir awesome
cd awesome
majin init
majin title "Legendary Adventure"
majin author "An Awesome Dude"

# create a main script with some code, save it as, say, scripts/main.js

majin main scripts/main.js
spherun .


A while back my plan was to repurpose Cell to do this, but I decided against it because an automated build tool will still be needed in the future, once TypeScript support, minification, etc. comes into play.  So instead I'm making the project management its own thing.
Title: Re: minisphere 4.3.8
Post by: Fat Cerberus on December 11, 2016, 01:50:47 am
So far, there are two overarching goals for the minisphere 5.0 release:

Title: Re: minisphere 4.3.8
Post by: Rhuan on December 27, 2016, 08:49:01 am
Can I ask what the idea behind Cell is? I never really liked the idea of spks  as I liked being able to see all the documents in my games and being able to use custom folder structures within them - maybe I was misunderstanding something but an spk didn't seem to add any value and potential made the above harder.

When I first saw mention of "the cell compiler" I thought this was going to be somehow compiling the JS to create massive optimisations but I assume a) I was wrong AND b) it's never going to do that?
Title: Re: minisphere 4.3.8
Post by: Fat Cerberus on December 27, 2016, 09:19:45 am
You don't have to make an SPK, Cell can also compile to a loose directory.

The idea is to automate Sphere package creation, whether that "package" is a directory or SPK.  One of the features planned for minisphere 5.0 is TypeScript support, but Duktape is too slow to do that on the fly, so the idea is to use Cell to compile the TypeScript to JS, minify scripts, etc.  In other words it's a build step.  That's completely optional of course - if you want to code directly in JS and in general use the supported formats directly, that's fine too. :)
Title: Re: minisphere 4.3.8
Post by: Fat Cerberus on December 27, 2016, 12:55:44 pm
Follow-up to the above: The full minisphere toolchain actually consists of a few executables:

Title: Re: minisphere 4.3.8
Post by: Rhuan on December 27, 2016, 04:58:25 pm
I should probably try and get sphererun compiled on mac next and then do cell and ssj - though I've got sidetracked beginning to script systems for a game now I have sphere working.

(Also I don't know what GDB is - remember until the last week or so it was a couple of years since I'd done any programming unless you count excel formulae... So in general please expect me to not know what lots of common modern programming/scripting tools are)
Title: Re: minisphere 4.3.8
Post by: Flying Jester on December 27, 2016, 06:37:41 pm
gdb is the GNU debugger. On Mac, you probably have lldb, the llvm debugger. Both can debug executables from either toolchain. Their syntax is not quite compatible (lldb tends to be more verbose, I far prefer gdb).
Title: Re: minisphere 4.3.8
Post by: Rhuan on December 28, 2016, 06:13:06 am
So I've been playing around a bit with miniSPhere - it's implementation of the 1.5 api seems to be missing a few of the graphic primitives:

1. FilledCircle - miniSPhere has surface_object.filledCircle but no vanilla Filled Circle (it has GradientCircle which can be used instead but is assumedly slightly slower)

2. Ellipse functions in general the 1.5 api lists quite a few of them both in the primatives section and the surfaces section, I can't see any in miniSphere's vanilla.c

I've probably missed one or more; but hey loops and pixel  operations get me there anyway... :P
Title: Re: minisphere 4.3.8
Post by: Fat Cerberus on December 28, 2016, 06:49:58 am
GradientCircle is probably not any slower in reality.  It's hardware accelerated and is drawn as a triangle fan in both the solid and gradient cases, so there's no real difference unlike with software rendering.  That said, the missing functions *are* very much a compatibility issue, so I'll add them.  Thanks for pointing them out. :)
Title: Re: minisphere 4.3.8
Post by: Rhuan on December 28, 2016, 11:14:55 am
Found another engine difference between 1.5 and miniSphere. Abort in 1.5 would parse new line instructions (/n) And would not say the exit was an error if /n was included in the text.

(Some games used this as the actual method of exiting the game and giving the user a goodbye message or something - obviously it's not a great method as it's giving them white on a black screen etc. so likely not going to be used in any serious project but hey)
Title: Re: minisphere 4.3.8
Post by: Fat Cerberus on December 28, 2016, 11:51:28 am
Yeah, I didn't bother emulating that Abort() behavior, it didn't really seem worth it.

I'll make a 4.3.9 release in a day or so to add the missing ellipse functions.  I can't do it immediately because the trunk is in shambles thanks to the ongoing Cell overhaul, so I'll have to make a branch off of 4.3.8 and cherry-pick the relevant fixes.
Title: Re: minisphere 4.3.8
Post by: Rhuan on December 29, 2016, 04:48:37 am
Another question... Possibly more a feature request.

Something I wanted to be able to do before was make hybrid custom data files so I could store mixed game resources in one document. The problem with this is Sphere does not support converting a byte array to an image or sound or vice versa.

The only way I could this before with images was to write the image to a file then open it as a rawfile, read the data and then write it into my data file - then to load it read the data, write it to a temporary file with an appropriate image extension then use the LoadImage function - there was no version I could do at all with sound.

I had been hoping to use a custom spriteset format in the project I'm working on but it looks like I probably can't as it will involve way too much disk input/output to load it - shouldn't there be a facility to convert a byte array to an image or other equivalent functions without having to write it to disc?
Title: Re: minisphere 4.3.8
Post by: Rhuan on December 29, 2016, 05:25:23 am
And got another issue for you; this time an error in miniSphere.... Sorry I have a tendency to use obscure functions.

Background: I used an online sprite generator which creates an image output containing the various frames of the sprite; as I'm intending to use it quite a bit AND it also includes some stuff at the bottom which prevents sphere's image to spriteset tool working I wanted to write a script that could read lots of outputs form this sprite generator and make them into sphere spritesets (or some other custom spriteset format though probably the former due to the point in my previous post).

However miniSphere can't do it quite right - Sphere 1.5 running under Wine can.

Method:
1. Load the image as a surface
2. use nested loops to slice the image up surface_object.cloneSection(x,y,w,h).createImage();
3. do stuff with the resultant images

Problems (in each case no problem in 1.5 but a problem in miniSphere):
1. whilst the image loads with its background being transparent initially after it's sliced up the background becomes opaque in miniSphere - Sphere 1.5 keeps it transparent
2. the createImage() function in minisphere somehow fails to clean its buffer - if I remove the createImage step and blit  a few of my outputs in miniSphere they look ok apart from the transparency issue BUT if I put the createImage step back each frame becomes the result of bliitng all previous frames on top of each other seemingly with transparent backgrounds except a non-transparent background is added to the whole thing at the end.

edit: and another point miniSphere doesn't have the CreateSpriteset function.
- so far miniSPhere can run all my game code but not utility code - but sphere 1.5 under wine can run the utility stuff so it's ok

later today I'll try and post a completeness check of the miniSphere functions as I seem to keep finding ones it's missing.
Title: Re: minisphere 4.3.8
Post by: Fat Cerberus on December 29, 2016, 09:22:55 am

I had been hoping to use a custom spriteset format in the project I'm working on but it looks like I probably can't as it will involve way too much disk input/output to load it - shouldn't there be a facility to convert a byte array to an image or other equivalent functions without having to write it to disc?


With the Sphere v1 API, you're right, there isn't a way to do this.  However, Sphere v2 does include such a thing:
https://github.com/fatcerberus/minisphere/blob/v4.3.8/docs/sphere2-api.txt#L769-L777

To get an ArrayBuffer from a file, use the FileStream API:
https://github.com/fatcerberus/minisphere/blob/v4.3.8/docs/sphere2-api.txt#L1088


For sounds, you could look into the SoundStream API, though it may not be an exact match to what you need:
https://github.com/fatcerberus/minisphere/blob/v4.3.8/docs/sphere2-api.txt#L1047-L1085

Sorry, but you're going to have to learn the new API at some point. ;)

I'll respond to your other post in a bit.
Title: Re: minisphere 4.3.8
Post by: Rhuan on December 29, 2016, 09:37:57 am
Thanks for the pointers... Not so sure on learning this new api, I know the old one so well that trying to replace it in my mind would be a lot of effort.

As for the other points, no rush - you've done an awesome job making a modern rebuild of sphere that generally runs faster and does stuff better whilst using a lot of code - the face that I can find a few bits that don't work at the moment isn't a massive issue.
Title: Re: minisphere 4.3.8
Post by: Fat Cerberus on December 29, 2016, 09:44:59 am

And got another issue for you; this time an error in miniSphere.... Sorry I have a tendency to use obscure functions.


That's fine, good to have someone testing the darker corners of the engine. :)  Keep in mind though: The Sphere v1 API is deprecated.  I'll maintain is so far as fixing compatibility issues and obvious bugs, but I won't go out of my way to accommodate it.  That having been said, moving on...

Quote

Background: I used an online sprite generator which creates an image output containing the various frames of the sprite; as I'm intending to use it quite a bit AND it also includes some stuff at the bottom which prevents sphere's image to spriteset tool working I wanted to write a script that could read lots of outputs form this sprite generator and make them into sphere spritesets (or some other custom spriteset format though probably the former due to the point in my previous post).

However miniSphere can't do it quite right - Sphere 1.5 running under Wine can.

Method:
1. Load the image as a surface
2. use nested loops to slice the image up surface_object.cloneSection(x,y,w,h).createImage();
3. do stuff with the resultant images

Problems (in each case no problem in 1.5 but a problem in miniSphere):
1. whilst the image loads with its background being transparent initially after it's sliced up the background becomes opaque in miniSphere - Sphere 1.5 keeps it transparent
2. the createImage() function in minisphere somehow fails to clean its buffer - if I remove the createImage step and blit  a few of my outputs in miniSphere they look ok apart from the transparency issue BUT if I put the createImage step back each frame becomes the result of bliitng all previous frames on top of each other seemingly with transparent backgrounds except a non-transparent background is added to the whole thing at the end.


Your issue with createImage "not clearing its buffer" seems weird.  ssSurface#createImage() does this:
https://github.com/fatcerberus/minisphere/blob/v4.3.8/src/engine/vanilla.c#L4169-L4182

There's no buffering involved - it just clones the contents of the surface (in fact surfaces and images are the same thing to the engine internals, it's only the API that's different).  So not sure what's causing that errant compositing behavior.  Seems weird.

Quote

edit: and another point miniSphere doesn't have the CreateSpriteset function.


CreateSpriteset() was left out on purpose, actually.  Best I can tell, how Sphere 1.x implements spritesets is by using the JavaScript spriteset object directly, without having a dedicated internal structure.  The JS engine and map engine are quite tightly coupled.  minisphere tries to decouple them as much as possible and has its own internal spriteset type, so getting the two domains (JS and C) to cooperate is harder.  I might be able to swing something though, I'll see what I can do when I have some spare time.  Thanks again for testing, it's appreciated! ;)
Title: Re: minisphere 4.3.8
Post by: Fat Cerberus on December 29, 2016, 12:45:53 pm
Looking at the code for cloneSection real quick, I think I might have an idea what causes that loss of transparency bug.  I'll check into it and fix it if so.
Title: Re: minisphere 4.3.8
Post by: Fat Cerberus on December 29, 2016, 02:32:51 pm
@Rhuan: I can't reproduce that transparency bug.  I have the following test code:
Code: (javascript) [Select]

const prim = require('prim');
var image = LoadSurface('Title.png');
var slice1 = image.cloneSection(0, 0, 300, 75).createImage();
var slice2 = image.cloneSection(0, 75, 300, 75).createImage();
while (system.run()) {
prim.fill(screen, Color.Chartreuse);
image.blit(0, 0);
slice1.blit(300, 0);
slice2.blit(300, 75);
screen.flip();
}


Which produces the screenshot attached.  As you can see, the image remains transparent even when sliced up (I didn't cheat by giving it a green chartreuse background ;)).
Title: Re: minisphere 4.3.8
Post by: Rhuan on December 30, 2016, 07:07:00 pm
So I ran it on my mac build of miniSPhere and got two problems....

Firstly duplication of the text and secondly blue background.

Note the difference between the two screenshots;

This code produced the one with the duplication:
Code: (javascript) [Select]
const prim = require('prim');
var image = LoadSurface('Title.png');
var slice1 = image.cloneSection(0, 0, 300, 75).createImage();
var slice2 = image.cloneSection(0, 75, 300, 75).createImage();
while (!IsKeyPressed(KEY_ESCAPE)) {
        prim.fill(screen, Color.Chartreuse);
        image.blit(0, 0);
        slice1.blit(300, 0);
        slice2.blit(300, 75);
        screen.flip();
}


This code prodcued the one without the duplication:
Code: (javascript) [Select]
const prim = require('prim');
var image = LoadSurface('Title.png');
var slice1 = image.cloneSection(0, 0, 300, 75);
var slice2 = image.cloneSection(0, 75, 300, 75);
while (!IsKeyPressed(KEY_ESCAPE)) {
        prim.fill(screen, Color.Chartreuse);
        image.blit(0, 0);
        slice1.blit(300, 0);
        slice2.blit(300, 75);
        screen.flip();
}


In both cases the transparency is lost.
Title: Re: minisphere 4.3.8
Post by: Fat Cerberus on December 31, 2016, 12:24:14 am
In both cases the output is the same here.

One thing I do notice is that in your screenshots, the blitted image is also smaller than it should be.  I wonder if there's a driver bug at play here...

edit: Wait a minute... does your Mac have a Retina display?  I vaguely remember that you have to do something in XCode to make your app DPI-aware, otherwise Allegro doesn't render things properly (OS X sneakily scales everything up but OpenGL doesn't realize this).

edit 2: See here: https://github.com/liballeg/allegro5/blob/master/README_macosx.txt#L25-L50
Title: Re: minisphere 4.3.8
Post by: Rhuan on December 31, 2016, 03:33:59 am
Ok So the image size was a mistake I'd made - I's downloaded the thumbnail rather than the actual image... Adjusting to the proper image gives the following two results.

The one with the half duplicated image is the one with the .createImage() calls in it the one where the only issue is the loss of transparency is without the .createImage() calls.

I do have a retina display but that's not relevant - it's only meant to cause problems if you move a window between the retina display and another screen - I haven't been using a second screen so can't test how this would work at the moment.

Edit: Found another issue, though this one is kinda minor. Savefile support, in sphere 1.5 if you write a boolean to a savefile it writes it as 0 for false and 1 for true it detects that it's meant to be a boolean from the default value you give it when loading. Whereas minisphere will write true and false, so any savefile with booleans written by one is not readable by the other.
Title: Re: minisphere 4.3.8
Post by: Fat Cerberus on December 31, 2016, 10:15:08 am
This bug makes no sense. ???  On all platforms including Windows, minisphere runs Allegro in OpenGL mode.  So the behavior should be identical, and as you could see it works as designed for me.  For some reason though, it doesn't seem to like your mac.  I'm very confused.
Title: Re: minisphere 4.3.8
Post by: Rhuan on December 31, 2016, 01:28:51 pm
Maybe I something I messed up in the compilation, seems super odd though yes.
Title: Re: minisphere 4.3.8
Post by: Fat Cerberus on January 01, 2017, 06:35:14 pm
I'm going to invest in a MacBook sometime in the next few months, so then I'll be able to support minisphere officially on all 3 major platforms.  So if this issue isn't solved by then, hopefully I'll be able to get to the bottom of it myself. :)

The only other alternative for me is to go the Hackintosh route, and I'm really not up for all the headaches that apparently entails.
Title: Re: minisphere 4.3.8
Post by: Rhuan on January 01, 2017, 07:25:51 pm
I hope you're not buying a macbook just for minisphere? They're a tad expensive...

I should be able to try and track down the bug at some point I haven't into it too much as I have ways of gettign around this - as said it may just be something I messed up on compilation - I built miniSphere and Allegro and most of the other dependencies from scratch using my own xcode project I didn't use any of the provided makefiiles so there's a lot of room for me to have got something wrong.
Title: Re: minisphere 4.3.8
Post by: Fat Cerberus on January 01, 2017, 07:36:51 pm
Yeah, I'm aware the price tag is pretty steep.  But I'd like to learn OS X and iOS development in general.  The last time I used a Mac was back in high school, and that was with OS 9 (most of time running At Ease).  Believe it or not I've never actually used OS X before!
Title: Re: minisphere 4.3.8
Post by: Eggbertx on January 02, 2017, 12:46:09 am
Is it possible to run OS X in Virtualbox or VMware? That might make things faster on the development side..
Title: Re: minisphere 4.3.8
Post by: Flying Jester on January 02, 2017, 01:25:51 am
A Mac Mini is also a cheaper option. But MacBooks really are pretty nice. They feel very solid, they have very good screens, keyboards, and touchpads, and have very decent built in speakers for a laptop.
Title: Re: minisphere 4.3.8
Post by: Eggbertx on January 02, 2017, 02:19:29 am
It was pretty nice, but why shell out a bunch of money for a macbook when I can buy (and I did) a used thinkpad?
I have no idea why, but thinkpads seem to sell for crazy cheap on ebay. I got this one for $~80 after shipping. My only complaint is that it has two really tiny speakers.
Title: Re: minisphere 4.3.8
Post by: Fat Cerberus on January 02, 2017, 11:20:56 am

Is it possible to run OS X in Virtualbox or VMware? That might make things faster on the development side..


I thought of this a while back, but VirtualBox only allows OS X VMs to run on OS X.  Don't know if the same applies to VMware.
Title: Re: minisphere 4.3.8
Post by: Flying Jester on January 02, 2017, 03:00:04 pm
That's because as a part of the license for OS X, you may only run it on an Apple-produced computer. This includes under a VM. And since VMWare and VirtualBox require OS support, not having emulation to fallback on like Qemu does, they have to play nice with Apple to have OS X support at all.
Title: Re: minisphere 4.3.8
Post by: Rhuan on January 02, 2017, 07:18:21 pm
You can get a second hand mac mini or macbook for fairly cheap if you just want any device that will run osx. If you're looking for a high spec machine to last for a long time then the price gets high fast though.
Title: Re: minisphere 4.3.8
Post by: Fat Cerberus on January 08, 2017, 06:16:47 pm
I just had an idea.  If I end up getting a Mac, I could compile minisphere for iOS.  Then maybe we could get Sphere into the App Store somehow.  I don't know what the process for that is, or if Apple would allow it, but it's worth looking into at least.

There shouldn't be any technical issue here anyway, Allegro supports iOS and I know several people have compiled Duktape for iOS too.
Title: Re: minisphere 4.3.8
Post by: Eggbertx on January 08, 2017, 07:48:54 pm

I just had an idea.  If I end up getting a Mac, I could compile minisphere for iOS.  Then maybe we could get Sphere into the App Store somehow.  I don't know what the process for that is, or if Apple would allow it, but it's worth looking into at least.

There shouldn't be any technical issue here anyway, Allegro supports iOS and I know several people have compiled Duktape for iOS too.

That sounds pretty cool. I could try compiling minisphere for Android, though I have no idea what the process is like.

Also,  I noticed that if a function throws an exception that isn't caught in a try/catch,  it aborts with "script not found `scripts/main.js`" despite the fact that scripts/main.js is the main script, and said function is in a library imported via require()
Title: Re: minisphere 4.3.8
Post by: Fat Cerberus on January 09, 2017, 12:15:25 pm

Also,  I noticed that if a function throws an exception that isn't caught in a try/catch,  it aborts with "script not found `scripts/main.js`" despite the fact that scripts/main.js is the main script, and said function is in a library imported via require()


I see, it only happens in the following case (all of these must be true):


I'll look into what causes that when I get a chance.  It is a bit odd but not really pressing to fix since it doesn't affect functionality (the game would have crashed in that case anyway).
Title: Re: minisphere 4.3.8
Post by: Eggbertx on January 09, 2017, 01:31:59 pm
Quote
An error is thrown from a module which is first required, either directly or indirectly, at the top of the main script

I always do my requires and script imports at the beginning, in the same way that you would do (most) #includes at the beginning of a main C source file.

Quote
The module throws an error before its corresponding require() has returned

I know that if there's a try/catch block, it doesn't have any issue. That was the idea. But the problem isn't that it throws an error but that said exception causes it to think that the main script doesn't exist, rather than giving information about where the exception was thrown.
Does a module have to return true (or false on failure) at the very end of the script?


Also also, is it possible to force a script/module to run synchronously? In my Chip's Challenge engine, it's having issues with skipping numbers, and it seems to be caused by that.
Title: Re: minisphere 4.3.8
Post by: Fat Cerberus on January 09, 2017, 02:28:18 pm
I know, I wasn't criticizing your methods, just summarizing the exact conditions that cause the bug for posterity.  I fully intend to fix it at some point. :)

Modules don't have to return anything.  The return value of require() is whatever the module assigns to module.exportsreturn statements are valid (because CommonJS modules are secretly self-executing functions), but will be ignored.

Not sure what you mean by running synchronously - require() is synchronous by definition; it only returns once the module has run to completion.  So you'll have to clarify here.
Title: Re: minisphere 4.3.8
Post by: Fat Cerberus on January 09, 2017, 02:46:03 pm
Seeing as quite a few bugs have been found now in minisphere 4.3.8, and 5.0 is still a ways off, I think what I'll do is get Cell back in working order without the TypeScript, etc. support yet, and release a 4.4.0 version with the bug fixes along with some recent enhancements (I have to review the commit history to figure out what these are...), this way there won't be a broken release up for months.

edit: Here's the full changelog so far for 4.4.0:
Code: (text) [Select]
* Adds `FS.readFile()` and `FS.writeFile()` which allow reading and writing
  an entire file in a single operation.

* Improves Sphere v1 compatibility by adding several missing APIs.

* Dispatch API functions will no longer accept a JavaScript source code string
  as a callback, the callback must now always be a function.

* Improves miniRT documentation.

* Fixes a bug where the miniRT terminal can be rendered before other threads.

* Fixes a bug where `random.sample()` sometimes returns `undefined`.

* Fixes a bug where error attribution doesn't work properly if `Duktape.act()`
  is not available at the time of the error.

* Fixes a bug where `CreatePerson()` won't accept a Spriteset object.
Title: Re: minisphere 4.3.8
Post by: Fat Cerberus on January 10, 2017, 11:36:58 am
Just finished implementing TypeScript support in Cell, so maybe minisphere 4.4.0 will get it after all :D

Code: [Select]

D:\src\spectacles-i>cell -bd dist
Cell X.X.X Sphere packaging compiler (x64)
the JavaScript-powered compiler for Sphere v2
(c) 2015-2017 Fat Cerberus

evaluating Cellscript.js...
processing 125 targets...
    transpile @/scripts/AIContext.js
    transpile @/scripts/Battle.js
    transpile @/scripts/BattleActor.js
    transpile @/scripts/battleAI/HeadlessHorseAI.js
    transpile @/scripts/battleAI/LumisquirrelAI.js
    transpile @/scripts/battleAI/Robert1AI.js
    transpile @/scripts/battleAI/Robert2AI.js
    transpile @/scripts/battleAI/ScottStarcrossAI.js
    transpile @/scripts/battleAI/ScottTempleAI.js
    transpile @/scripts/battleAI/VictorAI.js
    transpile @/scripts/BattleHUD.js
    transpile @/scripts/BattleScreen.js
    transpile @/scripts/BattleUnit.js
    transpile @/scripts/ConditionContext.js
    transpile @/scripts/Cutscenes.js
    transpile @/scripts/FieldMenu.js
    transpile @/scripts/gamedef/animations.js
    transpile @/scripts/gamedef/battles.js
    transpile @/scripts/gamedef/characters.js
    transpile @/scripts/gamedef/conditions.js
    transpile @/scripts/gamedef/game.js
    transpile @/scripts/gamedef/items.js
    transpile @/scripts/gamedef/maps.js
    transpile @/scripts/gamedef/math.js
    transpile @/scripts/gamedef/moveEffects.js
    transpile @/scripts/gamedef/scenes.js
    transpile @/scripts/gamedef/skills.js
    transpile @/scripts/gamedef/stats.js
    transpile @/scripts/gamedef/statuses.js
    transpile @/scripts/gamedef/text.js
    transpile @/scripts/gamedef/weapons.js
    transpile @/scripts/GameOverScreen.js
    transpile @/scripts/GrowthPopup.js
    transpile @/scripts/ItemUsable.js
    transpile @/scripts/LucidaClock.js
    transpile @/scripts/main.js
    transpile @/scripts/MapContext.js
    transpile @/scripts/maps/main.js
    transpile @/scripts/maps/Portentia.js
    transpile @/scripts/maps/Testville.js
    transpile @/scripts/MenuStrip.js
    transpile @/scripts/MoveMenu.js
    transpile @/scripts/MPGauge.js
    transpile @/scripts/MPPool.js
    transpile @/scripts/Party.js
    transpile @/scripts/PartyMember.js
    transpile @/scripts/Scrambler.js
    transpile @/scripts/Session.js
    transpile @/scripts/SkillUsable.js
    transpile @/scripts/SpecsClient.js
    transpile @/scripts/SpecsServer.js
    transpile @/scripts/SpriteImage.js
    transpile @/scripts/Stat.js
    transpile @/scripts/StatusContext.js
    transpile @/scripts/StoryManager.js
    transpile @/scripts/TargetMenu.js
    transpile @/scripts/testcases/brucesStory.js
    transpile @/scripts/testcases/lastLucidan.js
    transpile @/scripts/TestHarness.js
    transpile @/scripts/TitleScreen.js
    transpile @/scripts/TurnPreview.js
    transpile @/scripts/WeaponUsable.js
    install @/lib/kh2bar.js
    install @/lib/map/Atlas.js
    install @/lib/map/index.js
    install @/lib/map/Map.js
    install @/lib/map/ObstructionMap.js
    install @/lib/map/Person.js
    install @/lib/map/Tileset.js
    install @/images/GameOverScreen.png
    install @/images/Logos/GameLogo.png
    install @/images/Logos/TitleCard.png
    install @/images/ScottSucks.png
    install @/images/TestBattle.png
    install @/images/TitleScreen.png
    install @/maps/main.rmp
    install @/maps/Portentia.rmp
    install @/maps/Testville.rmp
    install @/maps/Portentia.rts
    install @/maps/TestvilleTiles.rts
    install @/music/Animals.ogg
    install @/music/BasicInstinct.ogg
    install @/music/BattleForLucida.ogg
    install @/music/BruceTellsHisStory.ogg
    install @/music/CreepFight.ogg
    install @/music/DeathComeNearMe.ogg
    install @/music/EndOfInnocence.ogg
    install @/music/GameOver.ogg
    install @/music/HymnOfLiberty.ogg
    install @/music/LamentForBruce.ogg
    install @/music/LucidaByNight.ogg
    install @/music/MalmagmaManor.ogg
    install @/music/ManorBoss.ogg
    install @/music/MyDreamsButADropOfFuel.ogg
    install @/music/MyDreamsButADropOfFuel2.ogg
    install @/music/OneWorldIsNotEnough.ogg
    install @/music/OurWorldCanStillBeSaved.ogg
    install @/music/Portentia.ogg
    install @/music/RiseOfThePrimus.ogg
    install @/music/RunawayTrain.ogg
    install @/music/ScottsHomecoming.ogg
    install @/music/SeasonsEnd.ogg
    install @/music/SpectaclesTheme.ogg
    install @/music/SubfloorManor.ogg
    install @/music/ThePromise.ogg
    install @/music/TheQuantumEnigma.ogg
    install @/music/TimeToLetGo.ogg
    install @/spritesets/battlers/Amanda.rss
    install @/spritesets/battlers/Bruce.rss
    install @/spritesets/battlers/Elysia.rss
    install @/spritesets/battlers/H. Horse.rss
    install @/spritesets/battlers/Justin.rss
    install @/spritesets/battlers/Katelyn.rss
    install @/spritesets/battlers/Lauren.rss
    install @/spritesets/battlers/Lumisquirrel.rss
    install @/spritesets/battlers/maggie.rss
    install @/spritesets/battlers/maggie_hippo.rss
    install @/spritesets/battlers/Robert.rss
    install @/spritesets/battlers/Scott T.rss
    install @/spritesets/battlers/Scott.rss
    install @/spritesets/battlers/Victor.rss
    install @/spritesets/battlers/Xemnas.rss
    install @/spritesets/invisible.rss
    install @/sounds/Munch.wav
    install @/icon.png
writing game.json... OK.
Title: Re: minisphere 4.3.8
Post by: Fat Cerberus on January 10, 2017, 01:18:12 pm

Also,  I noticed that if a function throws an exception that isn't caught in a try/catch, it aborts with "script not found `scripts/main.js`" despite the fact that scripts/main.js is the main script, and said function is in a library imported via require()


So I just looked into this bug.  By any chance do you happen to be throwing something which is not an instance of the JS Error class (or a subtype)?  If so I think I found the cause of the bug.  It's here:
https://github.com/fatcerberus/minisphere/blob/master/src/engine/script.c#L75-L76

When a script crashes, the script executor checks to see if there's an error on top of the Duktape stack, and if there isn't it pushes a generic "script not found" one (because it assumes that the script didn't run at all).  Trouble is, in JS, you can throw things that aren't Errors (for example, strings--something Sphere v1 code does a lot).  So the check is incorrect.  I'll need to refactor the script executor a bit to fix it, but it shouldn't be overly difficult.
Title: Re: minisphere 4.3.8
Post by: Fat Cerberus on January 12, 2017, 10:08:15 pm
I switched out TypeScript for Babel.  It turns out that, when run under Duktape with type checking turned on and a default type definition file (lib.d.ts), TypeScript compilation is SLOW.  That wouldn't be so bad by itself--Cell will only rebuild changed files--however with TypeScript all scripts have to be passed to the compiler simultaneously for types to work properly across files.  This means that even changing one file causes your entire game to be recompiled.  For a large game such as my own Spectacles, that comes out to several minutes worth of compilation.  Not nice.

I suspect that most people using the transpile() function in Cell would only be looking for ES6 syntax support anyway, and Babel works well for that.  It's also much less painful; scripts are parsed as ES6 modules, so the transpiler doesn't need to see the complete program upfront.  Each script is recompiled only when it changes.

Next order of business: Source map support.  When using a transpile step, the line numbers sometimes end up being way off under SSJ Blue (command-line SSJ is not an issue since it pulls the already transpiled source directly from the engine).
Title: Re: minisphere 4.3.8
Post by: Fat Cerberus on January 14, 2017, 01:28:06 am
I went the extra mile and added a Babel polyfill to the latest build of minisphere.  This allows game code to use new stuff like Map and Set.  So now, in combination with a Cellscript using the transpile module, minisphere can be a full ES6 environment :))
Title: Re: minisphere 4.3.8
Post by: Eggbertx on January 17, 2017, 11:26:44 am
In regards to the synchonous issue, I'm trying to figure out why it seems to be skipping numbers. If I replace system.sleep with GetKey, it only responds after 6 keypresses but still processes the n++ after every keypresse (so it displays 6,12,18,etc)
Code: [Select]

for(n = 0; n <999; n++) {
font.drawText(0,0,n);
//drawNumber(n,numbers_pos[0].x, numbers_pos[0].y);
screen.flip();
system.sleep(60);
}
Title: Re: minisphere 4.3.8
Post by: Fat Cerberus on January 17, 2017, 11:53:52 am
system.sleep() behavior changed in minisphere 4.3 - it now runs an update-render loop (because otherwise the frame count is meaningless).  Therefore if you have no render jobs set up using the Dispatch API, the screen will go blank (this is implied in the API documentation, but should be made clearer).  If you change it to Delay(1000); it works as expected.  Generally Sphere v2 code should use the Dispatch API (or the threads module, which wraps it).  Synchronous code should stick to the Sphere v1 API.

The bug of only updating every 6 numbers isn't with GetKey() by the way.  Changing to Delay() shows the same behavior: it goes 0, 6, 12, 18, etc.  It works properly if you change screen.flip() to FlipScreen().  So that's kind of weird and I'll have to look into it.
Title: Re: minisphere 4.3.8
Post by: Eggbertx on January 17, 2017, 01:43:35 pm
Yeah, there are some discrepancies between functions in the new API and the equivalent function from the old API.
And I didn't know about Delay()
Title: Re: minisphere 4.3.8
Post by: Fat Cerberus on January 17, 2017, 06:48:19 pm
I think the problem might be that screen.frameRate is set to 60 by default.  So minisphere tries to maintain 60fps but skips a bunch of frames because you stuck a 1s delay in there.  It's hardcoded not to skip more than 5 frames though, hence the 0->6->12 pattern.  FlipScreen is unthrottled by default (for compatibility reasons), so no frames get skipped.

You'd probably see the same behavior with Sphere 1.5 if you call SetFrameRate(60) first and do the delay manually.  In fact it might even be worse because Sphere 1.5 is programmed to skip up to 20 frames (!).

tl;dr: Your for loop works properly, but not all the frames are rendered.
Title: Re: minisphere 4.3.8
Post by: Eggbertx on January 19, 2017, 03:47:16 pm
Replacing screen.flip with FlipScreen worked. Maybe you should have a function or property to prevent throttling, or is there something like this already?
Title: Re: minisphere 4.3.8
Post by: Fat Cerberus on January 19, 2017, 04:14:15 pm
Code: (javascript) [Select]
screen.frameRate = Infinity;


That's actually documented:
https://github.com/fatcerberus/minisphere/blob/master/docs/sphere2-api.txt#L616-L619

Note that will peg the CPU though if you're rendering in a loop.
Title: Re: minisphere 4.3.8
Post by: Eggbertx on January 20, 2017, 02:56:44 am

Note that will peg the CPU though if you're rendering in a loop.

Is that a bug?
Title: Re: minisphere 4.3.8
Post by: Fat Cerberus on January 20, 2017, 10:14:43 am
No; by setting the framerate to infinity, you're asking the engine to draw as many frames as it possibly can.  That's why it's set to 60 by default, so the engine can sleep in between frames.
Title: Re: minisphere 4.3.8
Post by: Fat Cerberus on January 21, 2017, 03:07:19 am
What I might do for minisphere 4.4.0 is add a new property, screen.frameSkip.  You could then set that to 0 to prevent frames from being skipped, but still take advantage of throttling to keep your CPU usage under control.
Title: Re: minisphere 4.3.8
Post by: Fat Cerberus on January 27, 2017, 03:05:48 am
Good news: I'm finally done with the Cell overhaul and it's no longer a useless waste of disk space, but instead something I can be proud of. :))  Barring more bugs cropping up (hopefully I squashed them all this time!), minisphere 4.4.0 should be out in a few days with the new compiler.  Then everyone will be able to use ES6 goodies like arrow functions in their Sphere games!

Using Cell 4.4.0 and a properly-written Cellscript, it will already be possible to:


More Cell features will be coming in minisphere 5.0, like TypeScript compilation, so keep a look out for updates on that front. :)

As for minisphere itself: I tried very hard not to make any breaking changes to the Sphere v2 API, but I did ultimately have to make one change: Some of the FS functions got renamed.  For example, FS.unlink() is now FS.deleteFile().  One of Sphere's many strengths has always been its self-documenting API and using the POSIX names for file system functions wasn't doing us any favors in that department. ;)

tl;dr: minisphere 4.4.0 coming very soon!
Title: Re: minisphere 4.4.0
Post by: Fat Cerberus on January 27, 2017, 01:14:23 pm
After nearly two months of tireless work (minisphere unofficially sponsored by Monster Energy™ :P), the first minisphere release of 2017, v4.4.0, is out now!
Title: Re: minisphere 4.4.0
Post by: DaVince on January 27, 2017, 01:43:15 pm
Nice work, dude! :D
Title: Re: minisphere 4.4.0
Post by: Fat Cerberus on January 27, 2017, 03:28:18 pm
There's a bug in the Ubuntu packages which causes Cell to not be able to find require() modules.  Unfortunately Launchpad won't let me re-upload the packages without changing the version number. :(

Specifically the problem is that I didn't account for the makefile installing the system folder in /usr/share/minisphere, rather than next to the executables as in Windows.  That didn't show up when I tested it on Linux because I only tested the standalone build.  If I had tested 'make install' it would have shown up.  I'll remember to test that next time.

I'll post 4.4.1 tonight.
Title: Re: minisphere 4.4.0
Post by: Fat Cerberus on January 27, 2017, 05:00:29 pm
@Eggbert: Your PKGBUILD uses make install, so the Arch release probably suffers from the same bug. :(
Title: Re: minisphere 4.4.0
Post by: Eggbertx on January 27, 2017, 10:01:06 pm
It looks like games that use game.json cause minisphere to throw a parsing error and then segfault, while games that use the legacy game.sgm format work fine.
Title: Re: minisphere 4.4.0
Post by: Fat Cerberus on January 27, 2017, 10:05:47 pm
Huh, weird.  Is the game.json missing any properties?  Spectacles works fine for me and that uses game.json, but it's also compiled with Cell, which ensures everything is accounted for.  I'll look into it.
Title: Re: minisphere 4.4.0
Post by: Eggbertx on January 27, 2017, 10:14:19 pm
I don't think so, but just to verify that it isn't on my end, here (https://dl.dropboxusercontent.com/u/10266670/shmup.zip) is what I'm looking at. If I run `spherun .`, it fails, but it works fine from within shmup/dist/, the folder that Sphere Studio generated.
Title: Re: minisphere 4.4.0
Post by: Fat Cerberus on January 27, 2017, 11:04:13 pm
Yeah, Sphere Studio sets new projects to build in dist/ by default (this is also where Cell builds if you don't tell it otherwise).  As for your JSON file, there are a few issues with it actually.  The first one I noticed was:

Code: (json) [Select]

"main": "main.js",


SphereFS requires that path to be relative to game.json, or in other words it should be "scripts/main.js".  That's not what caused the crash, though.  The parse error you saw is caused by the trailing comma after the last property.  For some stupid reason, JSON doesn't support trailing commas.  Removing the last comma allows minisphere to load the game.

The last issue, which prevented the game from running, was that your manifest didn't specify v1.  Without that minisphere tries to run the game in v2 mode (basically: the main script is a module), which causes most legacy code to break (e.g. "personlib is not defined").  Here's a fixed JSON file that allows the game to run:

Code: (json) [Select]

{
"version": 1,
"name": "Unnamed Shump",
"author": "Eggbert",
"summary": "Follow Scott Starcross in his quest to stop the Primus.",
"resolution": "416x416",
"main": "scripts/main.js"
}


In any case I'll fix the segfault.  It definitely shouldn't crash, no matter how many errors are in the descriptor file. :)
Title: Re: minisphere 4.4.1
Post by: Fat Cerberus on January 28, 2017, 12:49:08 am
Okay, I uploaded 4.4.1 which makes Cell work in Linux again and also fixes the crash Eggbert found.
Title: Re: minisphere 4.4.2
Post by: Fat Cerberus on January 30, 2017, 01:33:41 am
4.4.2 fixes a Cell bug which caused it to SyntaxError on some Sphere 1.x code.  Basically transpile.v1() was running in permanent strict mode due to a missing option when invoking Babel.

edit: Nice, the minisphere thread made it to 100 pages!
Title: Re: minisphere 4.4.2
Post by: scojin on January 30, 2017, 06:01:51 pm
Quick question. I just downloaded v4.4.2(previously had 4.2) After tinkering with my cellscript, I got everything to build correctly, however, now I get a reference error thrown in the main script after declaring variables for character objects, like it's not reading my other scripts.
Is RequireScript() deprecated now? That is what I was using at the top of my main.js. Thx
Title: Re: minisphere 4.4.2
Post by: Fat Cerberus on January 30, 2017, 06:36:23 pm
RequireScript() is deprecated along with the rest of the Sphere v1 API.  However it should still work as designed.  In your Cellscript, are you using transpile.v1()?  If you just use transpile() then it will compile all your scripts as modules which might explain your issue.

edit: Now that I think of it some more, it could just be that your describe() doesn't include a version field.  Version defaults to 2; in a v2 game, the main script is loaded as a module.  In that case any globally declared variables are confined to the script that declared them.  You have to explicitly make them global then: global.whatever = "foo";  So your solution is to either do that, or add "version: 1" to describe().
Title: Re: minisphere 4.4.2
Post by: scojin on January 30, 2017, 10:36:02 pm
Okay, makes sense. I've been been using v2.
I've got my globals marked with global., so guess I'll have to study up on using modules.
Basically, I've got my scripts organized by the objects they are i.e. character.js has a character object and functions that go along with it(stat updates,etc), items.js has item objects, creates items, uses, etc., menu.js the menu and all of its functions, blah, blah

Since I know they're loaded as modules I can start trying to figure out how to properly structure everything.
I want to continue with the v2 to take advantage of all the capabilities, even though it's over my head right now.
Thanks for the quick reply.
Title: Re: minisphere 4.4.2
Post by: Fat Cerberus on January 31, 2017, 12:04:06 am
Good to see someone diving in to take advantage of all Sphere v2 has to offer :)

If you have a transpile() step in your Cellscript then you're all set to use the latest JS syntax, so you should read up on classes:
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Classes
Title: Re: minisphere 4.4.2
Post by: Fat Cerberus on February 01, 2017, 01:28:58 am
Just added support for ES6 syntax to Cell today.  I didn't originally consider it to be a big issue for ES6 not to be supported in Cellscripts, but it might be confusing when, say, destructuring assignment is supported in game scripts but not in the build script.  After all, I deliberately used JS for Cell so that it would be familiar for someone already writing Sphere code.
Title: Re: minisphere 4.4.2
Post by: Fat Cerberus on February 01, 2017, 10:56:10 am
Found out something interesting: While Babel does compile ES6 modules to CommonJS, and therefore you can import them using require(), the exports are all named.  Which means if you have a module with:

Code: (javascript) [Select]

// pig.js
export default
function getEaten() {
    console.log("You got eaten by the pig.  Good job.");
}


To load that module using require, you'd need to:
Code: (javascript) [Select]

const getEaten = require('./pig').default;


Apparently the same will be true in Node once they implement ES6 modules:
https://github.com/nodejs/node-eps/blob/master/002-es6-modules.md
Title: Re: minisphere 4.4.3
Post by: Fat Cerberus on February 02, 2017, 02:35:40 am
So I was thinking about my plans for minisphere 5.0 before, and I decided I want to do a rebranding for the next version - to "miniSphere" (with a capital S).  camelCase evokes JavaScript, plus it makes the name seem more... I don't know, substantial?  "Tiny size" is no longer one of the engine's main attractions (it's probably a bit bigger than Sphere 1.x, even)--in fact I actually stopped compiling it with size optimizations a while ago--so trying to convey that with the name doesn't make sense anymore.

miniSphere also seems like a better indication of the engine being a successor to the original Sphere.  I feel like having the name be all lowercase might have made it seem like a second-class citizen next to the original engine (which is thoroughly deprecated now, what with all the strides made since minisphere 4.x).  In other words people think it's an "alternative engine" rather than "the new Sphere on the block".  Clearly we would prefer them to think the latter rather than the former... ;)
Title: Re: miniSphere 4.4.3
Post by: Eggbertx on February 02, 2017, 11:46:08 am
What about NeoSphere, or something like that?
Title: Re: miniSphere 4.4.3
Post by: Eggbertx on February 02, 2017, 08:53:33 pm
Also, I just got an idea for QSI integration. Can you replace the CommonJS term module with something that would allow you to have the engine output and take input from the terminal? That way a developer could handle that without having to leave the IDE.
Title: Re: miniSphere 4.4.3
Post by: Fat Cerberus on February 02, 2017, 10:31:12 pm

Also, I just got an idea for QSI integration. Can you replace the CommonJS term module with something that would allow you to have the engine output and take input from the terminal? That way a developer could handle that without having to leave the IDE.


Not sure what you're getting at here.  You can print stuff to the (real) terminal using console.log() or one of its variants; there's no way to read from stdin though, that seems like something that's out of Sphere's jurisdiction?

edit: As for neoSphere, I considered it, but I don't think the name flows as well and besides I didn't really want to make a complete name change, so I settled for just tweaking the branding a bit.
Title: Re: miniSphere 4.4.3
Post by: Eggbertx on February 03, 2017, 12:01:42 am

Not sure what you're getting at here.  You can print stuff to the (real) terminal using console.log() or one of its variants; there's no way to read from stdin though, that seems like something that's out of Sphere's jurisdiction?


Why can't it use stdin? And my idea was that in the same way that you can do I/O from the term module, you could do the same, but with the system console or with QSI's console. I was planning on having QSI's console get console output from miniSphere anyway.
Title: Re: miniSphere 4.4.3
Post by: Fat Cerberus on February 03, 2017, 12:32:44 am

Why can't it use stdin?


No reason really (it's no less portable than, say, printf), but it just seems a bit too technical in nature to be part of the official high-level API (console.log() is only present because it's sometimes useful in the absence of a proper debugger).

The "term" module serves a different purpose anyway: It's there to make it easier for games to include an in-game cheat console, like a lot of FPS engines have.  It's not directly for the aid of the developer (it sometimes does help in that regard, admittedly).  I agree it would be useful if there were some SSJ integration with the term module though... just not sure what the best way to implement it is yet.
Title: Re: miniSphere 4.4.3
Post by: Fat Cerberus on February 04, 2017, 03:25:04 am
@Eggbert: You should update the AUR package, I've fixed several showstopper bugs in Cell since 4.4.0, including a nasty crash when files() doesn't find anything.  The current version also has support for ES6 syntax in Cellscripts, which wasn't in the initial release.
Title: Re: miniSphere 4.4.4
Post by: Fat Cerberus on February 06, 2017, 02:36:50 am
Posted miniSphere 4.4.4 to finalize the recent rebranding.  I also made a few minor tweaks, see the release notes on GitHub for the full changelog.
Title: Re: miniSphere 4.4.4
Post by: Fat Cerberus on February 08, 2017, 01:47:51 am
ES6 is awesome:
https://github.com/fatcerberus/spectacles-i/blob/master/lib/kh2Bar.js

Classes and default parameters are great to have (I wouldn't give them up for anything now), and import/export is much nicer than the module.exports finagling in CommonJS/Node-style modules.
Title: Re: miniSphere 4.4.4
Post by: Fat Cerberus on February 08, 2017, 01:02:01 pm
miniSphere 4.4.4 will probably be the last release for a while.  I have a long list of stuff I need to accomplish for v5.0, so I'd like to get started on that.  I don't like to do large refactorings in a patch cycle, so a 4.4.5 is unlikely to happen once I start adding 5.0 features to the codebase.

I originally wanted to implement ES6 support in the engine that works even without Cell, but Duktape is not fast enough to do transpilation on demand every time, so I'd need to implement a caching system or something.  So it's better to wait for the next major release to do that (see remark above on refactoring during a patch cycle).
Title: Re: miniSphere 4.4.4
Post by: FBnil on February 12, 2017, 02:46:48 pm
Thanks! Will take it for a spin.
Title: Re: miniSphere 4.4.4
Post by: Eggbertx on February 13, 2017, 07:39:36 pm
Now that QSI feels like it isn't going to implode with the slightest touch, I'm going to start getting into the nitty gritty stuff. So just to make sure, is miniSphere still fully compatible, and will it continue to be fully compatible with the legacy map engine, even for miniSphere generation projects?
Title: Re: miniSphere 4.4.4
Post by: Fat Cerberus on February 13, 2017, 10:27:44 pm

Now that QSI feels like it isn't going to implode with the slightest touch, I'm going to start getting into the nitty gritty stuff. So just to make sure, is miniSphere still fully compatible, and will it continue to be fully compatible with the legacy map engine, even for miniSphere generation projects?


Yes and yes.  I'm planning a new JS-based map engine for miniSphere 5.0, but the legacy functions and map engine aren't going anywhere for the foreseeable future.  The Vanilla API is considered deprecated in its entirety though--so don't expect any new features to be added to it, just bug fixes and compatibility tweaks where necessary.  All the work now is going into Sphere v2/Pegasus.
Title: Re: miniSphere 4.4.4
Post by: Eggbertx on February 13, 2017, 11:12:31 pm
Alright, awesome. I just didn't want to put a lot of effort into it to find out that it was on its way out.
And  minisphere/src/minisphere/map_engine.c has been a huge help so far :P
Title: Re: miniSphere 4.4.4
Post by: Fat Cerberus on February 14, 2017, 03:24:41 am
I started a repository for the first Sphere library designed specifically for use with miniSphere.  It's an overhaul of my Kingdom Hearts HP meter:
https://github.com/fatcerberus/kh2bar

I'll give it some proper documentation soon.  It also needs some updates to play more nicely with newer stuff like the Dispatch API, but it's certainly usable now for those so inclined.
Title: Re: miniSphere 4.4.4
Post by: Fat Cerberus on February 15, 2017, 02:32:41 am
Regarding that kh2Bar, here's a screenshot of it in action.
Title: Re: miniSphere 4.5.0
Post by: Fat Cerberus on February 20, 2017, 12:45:21 pm
Since I have a long laundry list of stuff to do for 5.0 and it's looking more and more like I won't make the March 28 deadline (i.e. second anniversary of v1.0), I decided to do an interim v4.5 release with all the minor changes I've made.  This version has a lot of small breaking changes to the API, so be sure to review the release notes before upgrading.  I think I'm finally getting close to finalizing the Sphere v2, so hopefully there won't be much more of this kind of thing needed.
Title: Re: miniSphere 4.5.2
Post by: Fat Cerberus on February 25, 2017, 03:02:01 am
miniSphere 4.5.2 adds support for the .mjs filename extension which will eventually be used in Node.js for ES6 modules.  Cell's behavior for transpile() has changed to match: script.js will get compiled as a normal script (which is Sphere 1.x compatible), while script.mjs gets compiled as a module and therefore supports the import and export keywords.
Title: Re: miniSphere 4.5.2
Post by: Fat Cerberus on February 27, 2017, 10:43:14 am
I have to say, I love ES6 in Sphere, even if it requires a build step for now.  Combined with all that Sphere v2 brings to the table, I haven't enjoyed writing Sphere code this much in a long time, and I've finally started working on my battle engine again.  I'm currently going through and converting all my constructor/prototype definitions to classes and having a blast doing it.  Classes, arrow functions, template strings, import/export... there's just so much awesome stuff!

Besides ES6, so far I would have to say my favorite Sphere v2 feature is named colors.  With Sphere 1.5, with color-heavy code, I always ended up getting overwhelmed with CreateColor() calls, because it's not always easy to visualize what color a given RGB triple represents without actually running the game.  With named colors, the X11 palette has so many options that there's almost always a predefined color that serves the purpose, and that makes for much more readable code.
Title: Re: miniSphere 4.5.2
Post by: DaVince on February 27, 2017, 01:37:27 pm
So JS modules get a different extension? Interesting. I hope this doesn't complicate things for text editors. (Syntax highlighting et al.)
Title: Re: miniSphere 4.5.2
Post by: Fat Cerberus on February 27, 2017, 07:04:45 pm
Yeah, that was what the Node.js team decided on.  The reason is that the semantics differ quite a lot between modules and regular scripts (automatic strict mode, global variables and functions are module-local, import/export aren't allowed in non-modules, etc.), so having a different extension lets tools be able to tell what mode to use to compile the code.  It helps me out too, since now the same transpile() Cellscript function can handle both Sphere v1 .js scripts as well as .mjs modules, which is awesome for migration.

There's a running joke in the JS community that .mjs stands for MichaelJacksonScript. :P

There is a problem with text editors not recognizing the extension as JS code, but I expect that will get better in time especially once a version of Node.js is available supporting JS modules.
Title: Re: miniSphere 4.5.2
Post by: Fat Cerberus on February 28, 2017, 12:03:18 pm
There are some bugs in the debugger when using transpiled ES6, where breakpoints don't always work.  I know of two cases where this happens for sure: when using generator functions, and class properties (get/set).  This happens because Babel's "retainLines" feature glitches out and appends the accessors to the end of the class instead of where they actually are in the source.  No code is placed on the lines where the property implementation originally is, so the breakpoint is never hit.  Unfortunately there's nothing I can do to work around the bug for now; I have to wait for a fix upstream.
Title: Re: miniSphere 4.5.3
Post by: Fat Cerberus on February 28, 2017, 08:24:27 pm
I added .mjs module support to Cell in the latest release (4.5.3).  Also fixed a bug which caused from.Object() to not enumerate any properties.
Title: Re: miniSphere 4.5.3
Post by: Fat Cerberus on March 04, 2017, 01:59:14 pm
For the next release I'm switching the default ES6 transpiler from Babel to TypeScript, since the latter is a lot faster.  TS is a strict superset of vanilla JS, so there shouldn't be any issue with the transition, but of course I'll make sure to do a lot of in-house testing before putting out a release with the change. ;)

By the way when I say "a lot faster" I mean a lot faster - build time for Spectacles went from what felt like about 30s down to 5s.  It's quite a dramatic difference.

TS with type-checking is a different story entirely - that is SLOW.

edit: Full rebuild of Radnen's Blockman takes ~13s on my Skylake i7 using TypeScript.  That took over a minute with Babel.

edit 2: Hm, that's unfortunate.  TypeScript will not convert ES6 generator functions to an ES5-compatible form; Babel does.  Generators are quite useful, e.g. as iterators.  It might worth the tradeoff for a faster build though... hm...  A future release of TypeScript will support downlevel transpilation of generators.  So this is a non-issue in the long run.
Title: Re: miniSphere 4.5.3
Post by: Fat Cerberus on March 05, 2017, 11:16:51 am
Hm, so the lack of generator support wasn't a dealbreaker for TS-as-default-compiler.  This, however, is:
https://www.typescriptlang.org/docs/handbook/iterators-and-generators.html

TypeScript purposely only transpiles for...of properly for arrays, not generic iterables (which is stupid since it's just a matter of calling collection[Symbol.iterator]() and stepping through it using .next().  Iterators are one of the most useful features in ES6, I don't want to lose them.  So Babel it is.  Hopefully it'll get faster in the future.
Title: Re: miniSphere 4.5.3
Post by: Eggbertx on March 05, 2017, 12:56:22 pm
I've been kind of neglecting it, but I finally got around to updating the AUR package.

Also, while syntax highlighting isn't complete yet, it will have TypeScript highlighting in addition to JavaScript highlighting.
Title: Re: miniSphere 4.5.3
Post by: Fat Cerberus on March 05, 2017, 01:09:41 pm

I've been kind of neglecting it, but I finally got around to updating the AUR package.

Also, while syntax highlighting isn't complete yet, it will have TypeScript highlighting in addition to JavaScript highlighting.


Thanks, good to know.  I'm probably going to revert to Babel as the default ES6 transpiler, but TypeScript support (i.e. .ts files) will be in the next release anyway, as an opt-in feature.  The main thing I'm trying to figure out is how to do incremental builds, because the TS compiler wants to see the complete program upfront.  So if even one file changes, everything has to be recompiled.
Title: Re: miniSphere 4.5.4
Post by: Fat Cerberus on March 05, 2017, 02:27:07 pm
New release.  4.5.4 has a small breaking change for Cell, the transpiler is now imported as:

Code: (javascript) [Select]

import { transpile } from 'compiler';


The rename was done in anticipating of future TypeScript support.  Calling TS a "transpiler" is stretching it since it's technically a different language, so I wanted a more generic name for the module than "transpile".  Note also that there's no longer a default export - transpile() is now a named export.

This release also adds support for for...of iteration of over from() queries.  Note that query results are enumerated as key/value pairs, NOT the raw values.
Title: Re: miniSphere 4.5.3
Post by: Eggbertx on March 05, 2017, 08:15:15 pm

The main thing I'm trying to figure out is how to do incremental builds, because the TS compiler wants to see the complete program upfront.  So if even one file changes, everything has to be recompiled.

Really? How long would it normally take to transpile it on a fairly large project?
Title: Re: miniSphere 4.5.3
Post by: Fat Cerberus on March 05, 2017, 08:56:53 pm

Really? How long would it normally take to transpile it on a fairly large project?


Yeah, the TS compiler wants to parse everything up front so it can resolve types, interfaces, etc. properly across files.  There's no separate compile and link steps like with C/C++ so you have to give it the full list of source files every time you build.  I haven't found a way around it.

Anyway, with type checking turned on and default definitions, Spectacles (https://github.com/fatcerberus/spectacles-i) takes 60+ seconds to compile if I remember right from the last time I tested it.  WITHOUT type checking it's actually pretty quick--around 5 seconds for the whole project--but that would still be annoying to wait 5s every time you go to test something.
Title: Re: miniSphere 4.5.4
Post by: Eggbertx on March 05, 2017, 09:06:37 pm
True, but miniSphere itself takes much longer to build. QtSphere IDE usually takes 5-6 seconds as well.
Title: Re: miniSphere 4.5.4
Post by: Fat Cerberus on March 08, 2017, 11:03:30 am
Good news: TypeScript recently merged full support for generators and for...of iteration when targeting ES5.  So I'll be able to make the switch after all and everyone can benefit from much faster ES6 transpilation times. :)

There will be some work needed to implement sourcemap support first though, before I can make a release using the TS transpiler (probably that'll be 5.0.0).  Sphere Studio can't currently debug TypeScript-compiled code properly because the line numbers don't match up.  SSJ can be used because it downloads source directly from miniSphere, but the mismatched line numbers still complicate setting up breakpoints for example.

edit: Holy crap, TypeScript has async/await!  That's awesome!
Title: Re: miniSphere 4.5.5
Post by: Fat Cerberus on March 10, 2017, 02:29:42 am
New version 4.5.5, bringing much faster transpile times thanks to TypeScript and fixing a few bugs in the process.  5.0 will bring more comprehensive TS integration; right now it's just being used for its transpiler.

I'm not sure how well TypeScript preserves line numbers in the transpiled code, so let me know of any issues with Sphere Studio debugging.  SSJ is fine in any case since it just downloads the transpiled code directly from the engine.  Sphere Studio tries to use the original source, so that might not match up now :(
Title: Re: miniSphere 4.5.5
Post by: Eggbertx on March 10, 2017, 07:35:13 pm
Is there relatively simple/straightforward documentation on Sphere/miniSphere's file formats (RSS, RMP, RTS, etc)?

also, looking at the GitHub repo source, what is obsmap?
Title: Re: miniSphere 4.5.5
Post by: Fat Cerberus on March 10, 2017, 10:04:30 pm
obsmap = obstruction map.  It handles collision detection for the map engine, which is not bounding-box but is instead based on intersecting line segments.  The obsmap keeps a list of all the obstructing line segments on a given map or tile.  You then pass in a rectangle to the obsmap representing the location of a spritebase, and it will tell you if the sprite is touching any of the segments.

For the file formats, miniSphere's implementation was pieced together by using the internal documentation for Sphere 1.x (https://github.com/sphere-group/sphere/tree/master/sphere/docs/internal), a bit of reverse engineering, and a lot of studying the Sphere 1.x source code.  The internal docs will honestly get you most of the way there, but there are some ambiguities you need to watch out for, particularly with the RSS format.  Ultimately, if all else fails, you can probably get a good idea of what to do by looking at the miniSphere source.  I tried to make the loaders as easy to follow as possible.
Title: Re: miniSphere 4.5.6
Post by: Fat Cerberus on March 11, 2017, 01:25:01 am
I've switched back to Babel for now, as the lack of sourcemap support makes debugging impossible in the IDE.  TypeScript tends to add a lot of boilerplate to support, e.g. generator functions, and also deletes blank lines, so the line numbers end up not matching up at all.  While Babel's line number retention isn't perfect either, it works well enough that debugging is still possible in 90% of cases.

TypeScript support will eventually be added back, but it's being postponed until miniSphere 5.0--which is still quite a ways off.
Title: Re: miniSphere 4.5.6
Post by: Eggbertx on March 11, 2017, 01:49:18 am
Would it be unrealistic for the IDE to calculate what the new line number would be and output the both TypeScript line number and the compiled Javascript line number?
Title: Re: miniSphere 4.5.6
Post by: Fat Cerberus on March 11, 2017, 01:56:08 am

Would it be unrealistic for the IDE to calculate what the new line number would be and output the both TypeScript line number and the compiled Javascript line number?


That's what source maps are for, and there's plenty of existing code to work with them.  The problem is that all the sourcemap libraries I was able to find only go one way, i.e. compiled->original.  Because I have to tell the engine where the breakpoints are relative to the compiled code (the code Duktape is actually running), I need a reverse lookup for that, i.e. original->compiled.  I'm sure something exists to do that, but I wasn't able to find it.  So since it may take me a while to get everything in order, I put Babel back for now so as not to compromise the debugging experience.
Title: Re: miniSphere 4.5.6
Post by: Fat Cerberus on March 11, 2017, 02:11:17 am
By the way there's an important change in 4.5.6 for from() queries.  Whereas before for...of over a query would enumerate key-value pairs, now it will only enumerate the values.  This change lets you keep a query around as if it were a "live array":

Code: (javascript) [Select]

evenNumbers = from(listOfNumbers).where(v => v % 2 === 0);

// print all the even numbers in listOfNumbers (live!)
for (let n of evenNumbers) {
    console.log(n);
}


That also lets you avoid wasting memory for a concrete array by alleviating the need for .select() in most cases.  Even better, by iterating through a query like that, you maintain the lazy evaluation behavior--as opposed to .select() which needs to run it to completion so it can return an array of the results.  In contrast, you can break out of the for...of early if needed, and the remainder of the results don't need to be processed.
Title: Re: miniSphere 4.5.6
Post by: Fat Cerberus on March 13, 2017, 02:34:15 am
It will soon be even easier to use the threads module when using ES6:

Code: (javascript) [Select]

import { Thread } from 'threads';

class SomeEntity extends Thread
{
    constructor()
    {
        super({ priority: 812 });
        // initialize entity
    }

    /* methods... */

    on_update()
    {
        /* update stuff... */
    }

    on_render()
    {
        /* render stuff... */
    }
}


Then:

Code: (javascript) [Select]

let myEntity = new SomeEntity();

myEntity.start();  // starts the thread running
myEntity.stop();   // kills the thread
myEntity.join();   // blocks until thread dies


No fiddling with thread IDs! :D
Title: Re: miniSphere 4.5.7
Post by: Fat Cerberus on March 14, 2017, 02:35:44 am
4.5.7 fixes the Sphere Studio project template so it compiles out of the box again.
Title: Re: miniSphere 4.5.7
Post by: Chad Zechs on March 14, 2017, 01:00:28 pm

4.5.7 fixes the Sphere Studio project template so it compiles out of the box again.

Is this in reference to it not finding the Cellscript file? That was my issue. I haven't had a chance to touch it since, mind you, but I was running into that.
Title: Re: miniSphere 4.5.7
Post by: Fat Cerberus on March 14, 2017, 01:40:03 pm
Yeah, related to the default Cellscript having an "import" statement, which now needs it to be a .mjs file.  The new project had it named as Cellscript.js which caused it to fail.
Title: Re: miniSphere 4.5.7
Post by: Chad Zechs on March 14, 2017, 01:42:49 pm
Agh, I did look at it and wonder if the filename was the issue, as I remember you mentioning the integration of .mjs. I didn't try it, though it couldn't have hurt.
Title: Re: miniSphere 4.5.7
Post by: Fat Cerberus on March 14, 2017, 10:25:53 pm
New branch policy for the Git repo: Development of the next version (right now planned as 5.0) with on "dev" branch, while "master" will be latest-stable.  Historically I've developed directly against the trunk, which is starting to delay major overhauls; by committing everything directly to master, patch releases only intended to fix bugs end up getting breaking changes too, so I end up holding off on doing them.  Which slows down development.
Title: Re: miniSphere 4.5.9
Post by: Fat Cerberus on March 31, 2017, 01:14:06 am
miniSphere 4.5.9 brings back TypeScript as the ES6 transpiler for much faster compile speeds.  Source map is appended to the end of the transpiled script and read by the SSJ plugin, so that debugging in Sphere Studio should now work properly.  Let me know of any glitches though, as I kind of rushed the implementation. :P
Title: Re: miniSphere 4.5.9
Post by: Beaker on March 31, 2017, 01:41:53 am
I'm in the process of porting The Rainis Manuscript over to miniSphere.  It seems very little is actually needed to get it to run, but I have noticed a few graphical bugs involving transparencies and fonts.  It's been a few months since I ran it last since I'm still working on the game proper, but I could try it again if these are not known issues.
Title: Re: miniSphere 4.5.9
Post by: Fat Cerberus on March 31, 2017, 02:12:04 am

I'm in the process of porting The Rainis Manuscript over to miniSphere.  It seems very little is actually needed to get it to run, but I have noticed a few graphical bugs involving transparencies and fonts.  It's been a few months since I ran it last since I'm still working on the game proper, but I could try it again if these are not known issues.


There indeed shouldn't be much effort needed to migrate - almost all of the Sphere 1.5 API is implemented.  I tried to make it as painless as possible to phase in the new features too - in large part you can mix Sphere v1 and v2 functions freely (objects are not interchangeable, however).

There's really only a handful of stuff that's not implemented, mostly some esoteric APIs like Complex() and a few surface blend modes (because they're impossible to implement with the OpenGL fixed-function pipeline), but otherwise everything should work correctly.  There shouldn't be any issues with fonts though; what kind of bugs did you encounter there?
Title: Re: miniSphere 4.5.9
Post by: Beaker on March 31, 2017, 03:02:34 am

There indeed shouldn't be much effort needed to migrate - almost all of the Sphere 1.5 API is implemented.  I tried to make it as painless as possible to phase in the new features too - in large part you can mix Sphere v1 and v2 functions freely (objects are not interchangeable, however).

There's really only a handful of stuff that's not implemented, mostly some esoteric APIs like Complex() and a few surface blend modes (because they're impossible to implement with the OpenGL fixed-function pipeline), but otherwise everything should work correctly.  There shouldn't be any issues with fonts though; what kind of bugs did you encounter there?


In the case of the font, it seems to either switch to a default font instead of the font loaded, or just not draw the transparencies of the font.  As far as I can remember, the only special things done to the font was possibly a color mask and doing the zoomblits.  In the case of the images, it seemed to be transparencies with zoomblits (basically, everything is a zoomblit).  Doing a quick re-look, I can't find the transparency with images problem, but I found a new bug wherein the restart(); function doesn't seem to reread the .sgm file like Sphere 1 does.  Sort of.  The first restart to change resolution works well enough (it doesn't work the same with the 320x240 option, but I assume that was deliberate), but the later restarts which were added to get around out-of-memory issues of (hopefully) yesteryear cause the engine to restart at the options menu instead of going to where it left off.

I also noticed some other differences as well with I think off-by-one pixels when drawing lines of text character by character vs drawing all the characters at once (a bit hard to explain, I'll have to send you a demo to explain that one).  When I finish the rest of the game, I can move over into doing the port fully, and give you a comprehensive list of the things I find.

In terms of the fonts, here's some screenshots:

Sphere:
(https://rpgmaker.net/media/content/users/94/locker/sphere1.png)

miniSphere:
(https://rpgmaker.net/media/content/users/94/locker/minisphere1.png)
Title: Re: miniSphere 4.5.9
Post by: Fat Cerberus on March 31, 2017, 03:13:47 am
I'll give this a better look tomorrow but you don't need to rewrite the .sgm file to change resolution in miniSphere (in fact the whole SGM format is deprecated, although Cell will still generate one to ease migration).  You can simply call screen.resize(w, h) to do that on-the-fly.
Title: Re: miniSphere 4.5.9
Post by: Fat Cerberus on March 31, 2017, 11:45:42 am
I think I found the font bug.  I'm assuming your fonts are RFNv1 (version = 1 in header)?  Sphere 1.x and miniSphere load RFNv1 fonts differently, due to a misinterpretation of the RFN spec on my part.

RFN specification:
https://github.com/sphere-group/sphere/blob/master/sphere/docs/internal/font.rfn.txt
It *does* say 255 is opaque while 0 is transparent, but I missed that part and only saw "8-bit grayscale".

Sphere 1.x:
https://github.com/sphere-group/sphere/blob/master/sphere/source/common/Font.cpp#L150-L153

miniSphere:
https://github.com/fatcerberus/minisphere/blob/master/src/minisphere/font.c#L136

In other words, Sphere 1 treats the single grayscale pixel as the alpha channel, while miniSphere treats it as a white level (which is normally what "grayscale" means).  So you lose your transparency as a result.  What I can't figure out is why the translucent pixels seem to be completely missing in your screenshot - if this is indeed the cause, then those pixels should be rendered, just opaque.

edit: Actually, I don't think that's the problem in this case (the bug still needs to be fixed of course)--if that were the issue then the font would have been rendered with a black background, which doesn't seem to be the case here.  So I'm not sure.  Could you maybe send me a copy of the .rfn font file so I could try to diagnose what's going on?
Title: Re: miniSphere 4.5.9
Post by: Fat Cerberus on March 31, 2017, 12:33:22 pm
Regarding that out-of-memory thing: That should no longer be an issue.  Sphere 1.x used to run out of memory all the time because it only initializes SpiderMonkey with a 5MB heap.  Duktape doesn't have that issue since it will just allocate memory directly as needed.  And its garbage collector is pretty aggressive (most objects are freed the second they fall out of scope).  So unless you have gigabytes of assets loaded simultaneously, I doubt you'll have an issue. :)
Title: Re: miniSphere 4.5.9
Post by: Beaker on April 01, 2017, 11:00:27 pm
It's a problem with drawZoomedText.  Using drawText will render correctly with the semi-transparent pixels, but drawZoomedText with zoom set to 2 or 1 will not draw them.

(https://rpgmaker.net/media/content/users/94/locker/font_example.png)
https://rpgmaker.net/users/Beaker/locker/main.rfn (https://rpgmaker.net/users/Beaker/locker/main.rfn)

Code: [Select]

// main.js automatically generated by Sphere Studio
// Vanilla compiler plugin
font1 = LoadFont("main.rfn");
font2 = LoadFont("main.rfn");
var c = CreateColor(255,0,128,255)

font2.setColorMask(c);

function game()
{
var txt = "SeT GQwWoO!";
while (true)
{
font1.drawZoomedText(0,0,2,txt);//renders without alpha
font2.drawZoomedText(0,20,2,txt);// ..

font1.drawZoomedText(0,40,1,txt);//also renders without alpha
font2.drawZoomedText(0,60,1, txt);// ...

font1.drawText(0,80,txt);//Render correctly
font2.drawText(0,100, txt);// ..

FlipScreen();
}
}

Title: Re: miniSphere 4.5.9
Post by: Fat Cerberus on April 02, 2017, 01:14:10 am
Interesting, thanks for looking into the issue.  I'll take a look at it when I'm not falling-over tired. :P
Title: Re: miniSphere 4.5.9
Post by: Fat Cerberus on April 02, 2017, 11:02:37 am

It's a problem with drawZoomedText.  Using drawText will render correctly with the semi-transparent pixels, but drawZoomedText with zoom set to 2 or 1 will not draw them.


I fixed it (on dev branch, patch coming soon).  drawZoomedText is implemented internally by rendering the text to a surface and then zoomblitting that to the screen.  The bug was caused by not setting the blend mode properly before drawing to the surface, causing the alpha channel to be lost.  This was a function that didn't get much in-house testing, since I think there was just one Sphere 1.x game I tested that actually used it (Blackfoot), and that was with the default font so the bug didn't show up.
Title: Re: miniSphere 4.5.10
Post by: Fat Cerberus on April 03, 2017, 12:40:18 am
Alright, 4.5.10 should fix the font rendering bugs.
Title: Re: miniSphere 4.5.10
Post by: Beaker on April 03, 2017, 07:24:52 pm
Good to know.  When I'm further along the development process for TRM, I'll send you a release to help debug some more issues.
Title: Re: miniSphere 4.5.11
Post by: Fat Cerberus on April 27, 2017, 01:50:39 am
4.5.11 fixes an annoying bug in the SSJ command-line debugger where entering an invalid command would cause it to repeat the previous action (especially frustrating if that previous action was a step or continue).  That's only supposed to happen if you leave the command line completely empty. :P  I also improved startup time a bit for Cell by using TypeScript to compile Cellscripts instead of Babel.
Title: Re: miniSphere 4.5.11
Post by: Fat Cerberus on May 01, 2017, 11:18:34 am
I'm pretty sure I'm going to drop the TypeScript support idea and just use TS as an ES6 transpiler, like it is now.  The more I read about TS, the more I dislike it, as I get the impression the type system is overengineered and at times too clever for its own good when it comes to type inference.  For example, the following is a compile error(!):

Code: (javascript) [Select]

const what = "pig";
if (what === "cow")
    console.log("the cow is going to eat you");
else
    console.log("the pig is going to eat you");


You get an error on line 2 because TypeScript treats the const not as a string but as a literal type: Its type is quite literally "pig".  Thus the comparison is an error because "cow" and "pig" are different types.

TypeScript also apparently uses static analysis in combination with type inference to narrow down types, which at first glance sounds convenient but (to me at least) makes me want to say "stop helping me!":
https://blog.mariusschulz.com/2016/09/30/typescript-2-0-control-flow-based-type-analysis

I can typecast it myself, thank you.

I like C# because I don't usually have to fight the type system - with TS narrowing types down to the level of individual values whenever it can, prototyping stuff becomes annoying in a lot of cases.  That kind of granularity is just not necessary.
Title: Re: miniSphere 4.5.11
Post by: Fat Cerberus on May 01, 2017, 02:13:39 pm
That said, as a transpiler TS is pretty incredible.  Given a little generator function:
Code: (javascript) [Select]

function* getEatenByThePig(numberOfTimes)
{
for (let i = 0; i < numberOfTimes; ++i)
yield `you got eaten by the pig for the ${i}th time`;
return 812;
}


TS creates this whole gnarly state machine for it:
Code: (javascript) [Select]

var __generator = (this && this.__generator) || function (thisArg, body) {
    var _ = { label: 0, sent: function() { if (t[0] & 1) throw t[1]; return t[1]; }, trys: [], ops: [] }, f, y, t, g;
    return g = { next: verb(0), "throw": verb(1), "return": verb(2) }, typeof Symbol === "function" && (g[Symbol.iterator] = function() { return this; }), g;
    function verb(n) { return function (v) { return step([n, v]); }; }
    function step(op) {
        if (f) throw new TypeError("Generator is already executing.");
        while (_) try {
            if (f = 1, y && (t = y[op[0] & 2 ? "return" : op[0] ? "throw" : "next"]) && !(t = t.call(y, op[1])).done) return t;
            if (y = 0, t) op = [0, t.value];
            switch (op[0]) {
                case 0: case 1: t = op; break;
                case 4: _.label++; return { value: op[1], done: false };
                case 5: _.label++; y = op[1]; op = [0]; continue;
                case 7: op = _.ops.pop(); _.trys.pop(); continue;
                default:
                    if (!(t = _.trys, t = t.length > 0 && t[t.length - 1]) && (op[0] === 6 || op[0] === 2)) { _ = 0; continue; }
                    if (op[0] === 3 && (!t || (op[1] > t[0] && op[1] < t[3]))) { _.label = op[1]; break; }
                    if (op[0] === 6 && _.label < t[1]) { _.label = t[1]; t = op; break; }
                    if (t && _.label < t[2]) { _.label = t[2]; _.ops.push(op); break; }
                    if (t[2]) _.ops.pop();
                    _.trys.pop(); continue;
            }
            op = body.call(thisArg, _);
        } catch (e) { op = [6, e]; y = 0; } finally { f = t = 0; }
        if (op[0] & 5) throw op[1]; return { value: op[0] ? op[1] : void 0, done: true };
    }
};

function getEatenByThePig(numberOfTimes) {
    var i;
    return __generator(this, function (_a) {
        switch (_a.label) {
            case 0:
                i = 0;
                _a.label = 1;
            case 1:
                if (!(i < numberOfTimes)) return [3 /*break*/, 4];
                return [4 /*yield*/, "you got eaten by the pig for the " + i + "th time"];
            case 2:
                _a.sent();
                _a.label = 3;
            case 3:
                ++i;
                return [3 /*break*/, 1];
            case 4: return [2 /*return*/, 812];
        }
    });
}


The fact that it was actually able to convert the for loop into a state machine is what really blew me away.
Title: Re: miniSphere 4.5.11
Post by: mezzoEmrys on May 01, 2017, 06:58:41 pm
What's the benefit of that state machine? It seems like a lot of code for a very simple closure generator?
Title: Re: miniSphere 4.5.11
Post by: Fat Cerberus on May 01, 2017, 09:52:09 pm
I assume it's so ridiculously complex because ES6 has a lot of very specific requirements for iterators and such and the transpiler is trying to cover all the possibilities.  You should see the TS/Babel output for a for/of loop.  That's pretty bad, too.
Title: Re: miniSphere 4.5.11
Post by: Flying Jester on May 01, 2017, 11:08:17 pm
If TypeScript actually got sum types/discriminated unions (as they claim they will), it might be very useful. But I think that's still not even agreed upon for inclusion.
Title: Re: miniSphere 4.5.11
Post by: Fat Cerberus on May 02, 2017, 12:06:57 am
Apparently that's a thing already:
https://www.typescriptlang.org/docs/handbook/advanced-types.html

Code: (javascript) [Select]

type Shape = Square | Rectangle | Circle;

function area(s: Shape) {
    switch (s.kind) {
        case "square": return s.size * s.size;
        case "rectangle": return s.height * s.width;
        case "circle": return Math.PI * s.radius ** 2;
    }
}


The compiler does some static analysis and automatically narrows the type in the switch cases, so that you can access the relevant members without an explicit type cast.  That's why I felt it was trying to be too clever for its own good, but I guess I can see the reasoning for it.
Title: Re: miniSphere 4.5.11
Post by: Rhuan on May 13, 2017, 07:33:53 am
Good to see that updates are still being made.

I'd like to do an updated build of miniSphere for mac, when would be a good time for me to grab the source?

(I.e. when no major changes are about to happen)
Title: Re: miniSphere 4.5.11
Post by: Fat Cerberus on May 13, 2017, 09:48:51 am
I already started (sporadically) working on miniSphere 5.0 so there are breaking changes on the trunk already, but you can build against the v4.5.11 tag ("git checkout v4.5.11")--or even just download the tarball:
https://github.com/fatcerberus/minisphere/releases/download/v4.5.11/minisphere-4.5.11.tar.gz

I have a pretty long backlog of stuff I want to implement for 5.0, so v4.5 will be current for quite a while still.
Title: Re: miniSphere 4.5.11
Post by: Rhuan on May 16, 2017, 04:43:56 pm
Found a nice feature of miniSphere that Sphere 1.5 doesn't have which is a symptom of the older JS engine used by 1.5; the JSON object and methods are available in miniSphere - I've been fiddling with Tiled map editor and with miniSphere I can save a Tiled Map as JSON and load it incredibly easily - sure I could write a script to load it a different way but 5 lines rather than 50 is nice :).
Title: Re: miniSphere 4.5.11
Post by: Rhuan on May 16, 2017, 08:11:42 pm
Ran into a not so nice feature of miniSphere for mac again - for some reasons when fiddling with images and particularly cutting up a surface you lose your alpha channel, Fat Cerberus, - I think we talked about this issue before but you could not reproduce it on a PC.

I'm wondering if it's a problem with Mac Allegro (or my compilation of Allegro such) though I can't think how to debug that at the moment.

In other news 4.5.11 runs with no noticeable differences I had to make the same two changes to the source I made before to get it to compile and run (rename the function panic and put in the path hack).
Title: Re: miniSphere 4.5.11
Post by: Fat Cerberus on May 16, 2017, 08:55:18 pm
There was a bug where fonts would be loaded without an alpha channel that I fixed, but nothing for images that I remember.  Could you give me some sample code that reproduces the bug?
Title: Re: miniSphere 4.5.11
Post by: Rhuan on May 17, 2017, 02:15:38 am

There was a bug where fonts would be loaded without an alpha channel that I fixed, but nothing for images that I remember.  Could you give me some sample code that reproduces the bug?
Correction it's only for surfaces, not images in general we discussed this in December and I could run a sript and get one result (with the error) and you ran the same script and got a different result (no error).

See discussion below:
http://forums.spheredev.org/index.php/topic,1215.1425.html

I could try and look into this later on, would be very useful for me to get it fixed - and it may well be something I've messed up inn the compilation.
Title: Re: miniSphere 4.5.11
Post by: Fat Cerberus on May 17, 2017, 10:36:52 am
There was a bug I fixed a few versions ago where Shape#draw() failed to normalize texture coordinates when drawn to a surface, which also indirectly affected prim.blit(), but yeah, I wasn't able to reproduce your bug in particular.  I also can't begin to figure out why it happens: You say it only affects cloneSurface and not cloneImage, but those two calls do the exact same thing internally (i.e. there is no low-level distinction between surfaces and images).
Title: Re: miniSphere 4.5.11
Post by: Rhuan on May 17, 2017, 04:10:12 pm
Found the fix to the error I'd found, did some googling...

But firstly to explain + clarify the issued:

1. Problems only arose if one used the Surface method: surface_object.cloneSection on a surface containing transparency - on using this anything that ought to be transparent in the clone would not be.

2. The problems would get worse if surface_object.createImage() was used on the cloned section returned from the above - weird seemingly inexplicable distortions would occur.

The result of my googling was this:
https://www.allegro.cc/forums/thread/616777
Apparently it would have been an issue on Linux too; the issue is with the function Allegro uses to create a new bitmap, the data for the new bitmap on Windows is automatically cleared whereas on Unix based systems it isn't.

Working fix I've implemented in my build:
In the file vanilla.c in the function: js_Surface_cloneSection(duk_context* ctx) I added  al_clear_to_color(al_map_rgba_f(0,0,0,0)); after the image is created. I've tested this with a couple of different test scripts and the error is now gone.

Full function with the addition:
Code: [Select]
static duk_ret_t
js_Surface_cloneSection(duk_context* ctx)
{
int      height;
image_t* image;
image_t* new_image;
int      width;
int      x;
int      y;

duk_push_this(ctx);
image = duk_require_class_obj(ctx, -1, "ssSurface");
x = duk_to_int(ctx, 0);
y = duk_to_int(ctx, 1);
width = duk_to_int(ctx, 2);
height = duk_to_int(ctx, 3);

if ((new_image = image_new(width, height)) == NULL)
duk_error_blame(ctx, -1, DUK_ERR_ERROR, "unable to create surface");
al_set_target_bitmap(image_bitmap(new_image));
al_clear_to_color(al_map_rgba_f(0,0,0,0));//the fix
al_draw_bitmap_region(image_bitmap(image), x, y, width, height, 0, 0, 0x0);
al_set_target_backbuffer(screen_display(g_screen));
duk_push_class_obj(ctx, "ssSurface", new_image);
return 1;
}

Title: Re: miniSphere 4.5.11
Post by: Fat Cerberus on May 17, 2017, 04:58:15 pm
Ah, that explains it, thanks for investigating!  Good to know the fix is simple.

I probably should have tested this in Linux; I have a Ubuntu VM I use when making releases but the extent of my testing there is usually just to make sure I didn't cause any major breakage (= if it doesn't crash, I'm satisfied).
Title: Re: miniSphere 4.5.11
Post by: Rhuan on May 18, 2017, 02:47:22 pm

Ah, that explains it, thanks for investigating!  Good to know the fix is simple.

I probably should have tested this in Linux; I have a Ubuntu VM I use when making releases but the extent of my testing there is usually just to make sure I didn't cause any major breakage (= if it doesn't crash, I'm satisfied).
I'll assume you'll add the above to the source tree?

Other question - I remember asking a while ago about the "CreateSpriteset()" function which doesn't exist in miniSphere - I can probably fake it by having a blank spriteset document I load would be nice to have it available to avoid the extra document. Another related function that doesn't seem to be there is spriteset_object.save() - I guess I could do what I'm doing for maps, have an image and a JSON file, load the JSON then load the image and have a function that sets up the spriteset object I need by combing info from the two but as Sphere's native spritesets do everything I need here nice if I could have those.

Alternatively I guess I could implement creating/saving spritesets in javascript with rawfiles but I image that would be more painful than it being implemented in C.

You said before that there was something unpleasant about the code behind the spriteset object and so the omission of the creation function was intentional?
Title: Re: miniSphere 4.5.11
Post by: Fat Cerberus on May 19, 2017, 12:31:08 am

I'll assume you'll add the above to the source tree?


Maybe not that exact patch; I'm thinking it might be better to move the clear call into image_new() to prevent the bug entirely.

As for CreateSpriteset, here's the issue: In miniSphere, a Spriteset object on the JS side is just a stand-in for a pointer on the C side, which points to a struct representing the actual spriteset data.  It's impossible (at least in Duktape) to catch all modifications to the JS Spriteset object; for example while one can easily swap out existing images and be sure it will work (because each image list entry has a custom setter):

Code: (javascript) [Select]

spriteset.images[0] = someOtherImage;


Things like adding and deleting images on the fly will NOT work and may cause strange things to happen:
Code: (javascript) [Select]

spriteset.images.push(someNewImage);  // suppose this becomes the 10th image (i.e. slot 9)
spriteset.directions[0].frames[0].index = 9;


If you try to use such a modified spriteset later, it will probably crash the engine because it didn't see the .push() and therefore has no idea about the new image(s) that you added.  So with that in mind, something like CreateSpriteset, specifically designed to build spritesets on-the-fly, just isn't going to work.  The reason all this works in 1.5, so my theory goes, is that the map engine just uses the JS spriteset object rather than using a pointer.  In other words it just reads directly from spriteset.images etc. so anything you add is automatically accounted for.  The downside is that doing things this way doesn't catch errors as quickly because the engine can't validate the modifications until you actually go to use the spriteset.

I can fix the spriteset system to be v1-compatible of course, but at this point it'd be a LOT of work for questionable gain, particularly as miniSphere 5.0 is slated to get a new modern map engine anyway and therefore the old map engine is about to become as deprecated as the rest of the v1 API.  Hm...
Title: Re: miniSphere 4.5.11
Post by: Fat Cerberus on May 19, 2017, 10:55:36 am
@Rhuan: Okay, the Surface#cloneSection() bug is fixed and will be in the next release, thanks again for that patch! ;D  I also fixed a similar bug in Surface#rotate() that I happened to notice along the way.
Title: Re: miniSphere 4.5.11
Post by: Rhuan on May 19, 2017, 12:45:57 pm
Thanks :) (Have you fixed it for surface.cloneSection as well as Surface#cloneSection?)

Couple of things I'd like to query:

1. Can you sell me on Sphere version2 - I'm still not sure what I think of it from the things you've said, I love the increased speed of miniSphere over 1.5 for most things, but can't currently see any significant benefits to changing API, I'm open to being persuaded though... I'm particularly interested in:
- understanding how Galileo is meant to work - my current graphics activities involve a lot of odd stuff - the code base for my current project uses quite a few graphics primitives and lots of zoomBlit and transformBlitMask in order to get the sort of graphical functions I want.
- Understanding what benefit the Dispatch API would have over the while loop of nested switch statements that I use - if any, with my loop everything is set up in order, and the switch statements ensure that the correct state is maintained and the correct actions happen, would Dispatch simplify this somehow? Could it be faster? i.e. instead of changing a state variable so the next iteration of the loop drops into a different case, I'd cancel one recurring job with dispatch and schedule another - should there be any particular benefit here?

2. On the Spritesets point
- I looked at the existing Sphere Map engine for my current project and quickly discarded it - it doesn't have nearly the level of flexibility I wanted
- So I scripted my own which is working very nicely (I can zoom in and out on the fly, move around, optimise by only drawing sprites that are on screen and handle various other things that my project wants); I'm therefore unlikely to be interested in the new map engine you're promising, at least for now.
- So why Sphere Spritesets - I want a way of having sprites that I can use easily, they need arrays of images, and directions and frames, all things that the sphere spriteset objects have
- I want to be able to load them quickly with one command rather than having a JS loop that goes through opening lots of images OR slicing up one larger image into the separate frames (I have a total of 202 images per sprite character - I split them accross two sprite files as some are a different size)
- ultimately I'm using tranformBlitMask to draw them when I need them, not any of Sphere's native sprite functions

- I don't know of a way to load an image in Sphere apart from via a spriteset or an image file (or a surface that I then slice up)

- I am using a non-sphere tool to make the spriteset graphics (it outputs a single image file with all the various frames)
- so my current process is to use the below setup code to create spriteset files (my project has a config document that stores if setup is complete or not - if not the below and a few other features are called, though for now setup only runs in 1.5 and the rest of the project only runs in miniSphere...

Code: [Select]
function make_sprite(name)
{
  var input  = LoadSurface("input/"+name+".png");
 
  //make the spritesets
  var output = CreateSpriteset(64, 64, 178, 21, 6);
  var a_output = CreateSpriteset(192, 192, 24, 4, 6);
 
  var t_x    = 0;
  var y      = 0;
  var x      = 0;
  var i      = 0;

  for (var y = 0; y < 21; ++y)
  {
    if(y < 4)
    {
      t_x = 7;
    }
    else if (y < 8)
    {
      t_x = 8;
    }
    else if (y < 12)
    {
      t_x = 9;
    }
    else if (y < 16)
    {
      t_x = 6;
    }
    else if (y < 20)
    {
      t_x = 13;
    }
    else
    {
      t_x = 6;
    }
    for(x = 0; x < t_x; ++x, ++i)
    {
      output.images[i]=(input.cloneSection(x*64,y*64,64,64).createImage());
      output.directions[y].frames[x].index = i;
    }
    output.directions[y].frames.length = x;
  }
  for(y = 0, i = 0; y < 4; ++ y)
  {
    for (x = 0; x < 6; ++ x, ++ i)
    {
      a_output.images[i] = input.cloneSection(x*192,1344+y*192,192,192).createImage();
      a_output.directions[y].frames[x].index = i;
    }
  }
 
  //name the directions - based on format of sheet
  output.directions[0].name   = "spell_n";
  output.directions[1].name   = "spell_w";
  output.directions[2].name   = "spell_s";
  output.directions[3].name   = "spell_e";
  output.directions[4].name   = "thrust_n";
  output.directions[5].name   = "thrust_w";
  output.directions[6].name   = "thrust_s";
  output.directions[7].name   = "thrust_e";
  output.directions[8].name   = "North";
  output.directions[9].name   = "West";
  output.directions[10].name  = "South";
  output.directions[11].name  = "East";
  output.directions[12].name  = "slash_n";
  output.directions[13].name  = "slash_w";
  output.directions[14].name  = "slash_s";
  output.directions[15].name  = "slash_e";
  output.directions[16].name  = "shoot_n";
  output.directions[17].name  = "shoot_w";
  output.directions[18].name  = "shoot_s";
  output.directions[19].name  = "shoot_e";
  output.directions[20].name  = "death";
  //attack animation directions
  a_output.directions[0].name = "North";
  a_output.directions[1].name = "West";
  a_output.directions[2].name = "South";
  a_output.directions[3].name = "East";
 
  output.save(name+".rss");
  a_output.save(name+"_a.rss");
}

var to_make = GetFileList("images/input");
for(var i = 0; i < to_make.length;++i)
{
  if(to_make[i][0]!=".")
  {
    make_sprite(to_make[i].slice(0,to_make[i].length-4));
  }
}
Title: Re: miniSphere 4.5.11
Post by: Fat Cerberus on May 19, 2017, 02:47:54 pm

Thanks :) (Have you fixed it for surface.cloneSection as well as Surface#cloneSection?)


Those are the same thing - the # is just shorthand for prototype, in other words "Surface#cloneSection" is an abbreviation for "Surface.prototype.cloneSection" (which is then inherited by all surface objects).

I'll address your other points later, once I have more free time to sit down and reply.
Title: Re: miniSphere 4.5.11
Post by: Rhuan on May 19, 2017, 03:12:14 pm


Thanks :) (Have you fixed it for surface.cloneSection as well as Surface#cloneSection?)


Those are the same thing - the # is just shorthand for prototype, in other words "Surface#cloneSection" is an abbreviation for "Surface.prototype.cloneSection" (which is then inherited by all surface objects).

I'll address your other points later, once I have more free time to sit down and reply.
OH.... When I'd previously been reading the v2 api I genuinely thought using it would involve # symbols all over my code...
Title: Re: miniSphere 4.5.11
Post by: Fat Cerberus on May 20, 2017, 11:10:04 am

OH.... When I'd previously been reading the v2 api I genuinely thought using it would involve # symbols all over my code...


Nope, it's just a documentation convention used in the JS community, "Class#method()", means if you have an object of type Class, you can call obj.method().  Whereas if I said instead "Class.method()" that means you call it exactly as written (it's a method of the class itself, i.e. a static method).

Now for your other points:


Quote
so my current process is to use the below setup code to create spriteset files (my project has a config document that stores if setup is complete or not


Hehe, these types of setup routines were the bane of my existence during early miniSphere development. ;)  "In the Steps of the Blackfoot" uses a similar process that involves slicing up an image containing a bunch of mushroom sprites and saving each one as an individual image file.  That forced me to implement Image#save() to be compatible.  Actually that was supposed to be the purpose of Cell when I first came up with it, to avoid the need for such first-time setup.  All the setup would instead go into your Cellscript, where the developer could trigger the setup process before distributing the game, but otherwise keep things in their "native" format for development.  Unfortunately, Cell still doesn't have all the routines necessary to do such a thing, it mostly only serves as a transpiler to allow you to use the new JS syntax features.  I'm working on expanding it though. :)
Title: Re: miniSphere 4.5.11
Post by: Rhuan on May 20, 2017, 01:09:13 pm
So:

Galileo sounds good for a few things I do e.g. I have a pointer that is currently drawn by calling Line() 160 times each frame - I may actually be able to optimise this just be drawing it onto a surface once and going from there though without switching to Galileo. Where I'm not sure how Galileo would work is for things like my sprite handing system - this checks and updates the direction, frame and location for any number of sprites then draws the ones that are on screen - as a given frame/direction combination is a different image I assume I'd have to re-texture a shape every time I wanted to draw - this would at a minimum double the function calls for drawing these - unless it's meant to work some completely different way, any thoughts on this?

Dispatch could make my code look a lot cleaner by the sound of it - and cut down the number of conditions being checked - though thinking about it I may be able to do the same by implementing a function queue and a loop - I assume that's effectively what dispatch is though?

Sphere v1s API is plenty flexible if you don't mind doing some odd stuff :P


Question on the spriteset setup point: as I can load a tempalte spriteset now I don't really need CreateSpriteset, the one thing I really do need is Spriteset#save(), any chance of this being implement OR.... I'll probably go read up on the file format and write a script to do it with raw files, shouldn't take too long...
Title: Re: miniSphere 4.5.11
Post by: Fat Cerberus on May 20, 2017, 01:45:59 pm
To be honest Dispatch was mostly implemented in service to Oozaru, the browser version of Sphere I eventually need to get around to implementing. :P  In a browser, the normal Sphere workflow of having a central game loop won't fly--the browser expects your script to finish promptly and will forcibly kill your loop if it runs too long.  So if you want to be web-compatible, you HAVE to use the Dispatch API and your code ends up looking like this:
https://github.com/fatcerberus/minisphere/blob/master/assets/template/src/main.mjs

There's no loop, the engine (or browser) provides it.  This is also how Node.js works, incidentally.

Regarding Galileo for sprites: Even if you have one Shape per sprite pose and have to draw them individually, it still (probably) ends up being faster than a normal blit--assuming you construct the Shapes in advance and just swap them out at render time--because the vertices are already uploaded to the graphics card.  Calling Image#blit() involves sending two triangles to the GPU each time.  Basically, the Sphere v1 API was designed in a time when computers could actually work at the pixel level and render in 2D.  Nowadays, everything is 3D and polygon-based.  You don't draw images directly, but rather draw a quad (two triangles) using the desired image as a texture.

To get an idea of how much you can save on overhead, take your line segments.  Each one is in fact rendered as two triangles:
https://github.com/fatcerberus/minisphere/blob/master/assets/system/modules/prim.js#L79-L98

As you can see there, every line involves uploading 4 (not 2!) vertices to the GPU.  If you can find a way to construct such shapes in advance, that upload only has to happen once (hint: you can draw it at different positions using a Matrix).  Circle/ellipses are even worse as they consist of a large number of vertices proportional to their radius.  At some point uploading vertices every single frame is going to become a major bottleneck, and that's why Galileo exists.  Basically, it gives you ultimate control over HOW things are drawn.  You can choose to draw on-the-fly, as you are now, are construct things in advance if that turns out to be better.  You don't have the latter option with v1.

So yeah, I rambled a bit, but hopefully you can understand the purpose now.

At this point I feel I should mention: Oozaru will not support the v1 API.  So me referring to it as deprecated is not just blowing hot air: While it's provided in miniSphere for backward compatibility to make migration easier, I'd no longer consider it part of the "Sphere standard" so to speak.  v2 is the way forward, for better or worse.  Hopefully better, though. :P
Title: Re: miniSphere 4.5.11
Post by: Rhuan on May 20, 2017, 02:43:36 pm
I've got no current interest in making a project that runs in a web browser, maybe I should but I just don't right now - so the oozaru support point isn't a key concern for me.

I'm using miniSphere because it runs faster than Sphere 1.5 and it compiled fairly easily for mac - if the v1 api is dropped from newer versions of it at some point I can always stick with the last version that supports it.

On the sprites point, let me see if I understand, so currently each sprite is comprised of approximately 240 images. right now I have code that loops through the array of characters and for each one determines which image from its sprite to draw, where to draw it, what colour mask to use and how zoomed in to make it - it then draws it and moves on to the next one.

Now lets say instead I made 240 textured shapes per sprite and the code worked out which shape to draw and where and then drew; script wise the difference would be the extra step of making the shapes and then the step of transforming the relevant shapes each frame to move them before drawing them.

Galileo certainly sounds better for any complex shapes comprised of multiple primatives or the like OR even for when it is particular image or shape you're intending to keep on screen a lot and move around - it doesn't seem to make sense for sprites though unless I'm misunderstanding it.
Title: Re: miniSphere 4.5.11
Post by: Fat Cerberus on May 20, 2017, 03:16:38 pm
I think I see the problem: You're looking at Galileo as a direct alternative to the classic primitives when that's not what it is at all.

Here's the thing, the Sphere v2 API is deliberately designed to be low-level.  So essentially what Galileo does is to expose how things already work internally to give you more control when you need it.  If you don't need that kind of control, the normal primitives are still just as serviceable as ever (so don't worry here, the "prim" module is NOT a second-class citizen, but just at a higher level of abstraction).  If that's not flexible enough--or performs poorly--*then* you have the option to step down a level and use Galileo to communicate directly with the engine on its own terms.

Let me put it this way: The entirety of the Sphere v1 API could have been built on top of Sphere v2.  The other way around, not so much.  You have strictly more options than before, not less. :D
Title: Re: miniSphere 4.5.11
Post by: Fat Cerberus on May 20, 2017, 03:26:48 pm
I guess the best way I can illustrate the point above is this: When you blit() an image for example, the engine literally does this every time:
https://github.com/fatcerberus/minisphere/blob/master/assets/system/modules/prim.js#L23-L38

What Sphere v2 gives you the option is to do most of those steps in advance.  It's up to you to decide when it's appropriate to do so, just don't make the mistake of thinking calling a bunch of blit() calls every frame is going to be magically faster than creating all those textured quads manually on the fly.  It won't be--because that's exactly what the engine is going to do on your behalf.
Title: Re: miniSphere 4.5.11
Post by: Rhuan on May 20, 2017, 04:00:50 pm

What Sphere v2 gives you the option is to do most of those steps in advance.  It's up to you to decide when it's appropriate to do so, just don't make the mistake of thinking calling a bunch of blit() calls every frame is going to be magically faster than creating all those textured quads manually on the fly.  It won't be--because that's exactly what the engine is going to do on your behalf.
I guess I'd think that having those low level steps done in the compiled engine rather than with extra calls from a script may be slightly faster?

I was specifically looking at this from the point of view of trying to see how reworking my graphics handling to use Galileo functionality could optimise/improve it - and couldn't see that - I see what you mean about lower level control though.

PS: I've got my setup running in miniSphere now thanks to using template spritesets instead of CreateSpriteset and writing my own spriteset saving function: http://forums.spheredev.org/index.php/topic,1440.msg8805.html
Title: Re: miniSphere 4.5.11
Post by: Fat Cerberus on May 20, 2017, 08:16:13 pm

I guess I'd think that having those low level steps done in the compiled engine rather than with extra calls from a script may be slightly faster?


Maybe, but probably not enough to matter in most cases.  In practice you only need to draw 60 frames per second; even with an interpreted JS engine like Duktape the extra overhead of bytecode execution isn't a huge issue.  As long as you can get all your rendering done in under 16.66ms (this is an eternity to the CPU), you're good.
Title: Re: miniSphere 4.5.11
Post by: Rhuan on May 21, 2017, 06:38:42 pm
Got another question for you on unsupported functions...

LoadSoundEffect - you said here http://forums.spheredev.org/index.php/topic,1434.0.html that it was 1.6 only; however it's actually in 1.5, I just tested it using 1.5 from the downloads drive.

It does largely overlap with LoadSound with one key difference the ability to play the effect multiple times overlapping each other, 1.5 actually had two ways of letting you do this easily either with LoadSoundEffect or with sound_object.clone().play() - neither of which you support.

I also tried to simulate this by creating new Mixers with your 2.0 api but I couldn't get any sound out at all either using just var sound = new Sound("sounds/file.ogg"); then sound.play(); or creating a Mixer and specifying it as a parameter, Sphere didn't crash but I didn't hear anything (and yes my volume was on, replacing new Sound with LoadSound meant I'd hear at least the first time it was meant to play).

Assuming audalis can be made to output somehting I'm happy to find a 2.0 way of doing this... (Though the api doesn't give me one thing I'd want to have - a way to either a) check if Mixer is producing sound or b) clone a sound object; either of these methods would allow me to do sound effects the way I want by either doing:

Code: [Select]
if(is_making_sound(mixers[mixers.length-1]))
{
  mixers.push(newMixer(blah,bla));
}

sound.play(mixers[mixers.length-1]);


OR

Code: [Select]
sound.clone().play()


(I note that at the moment as the v2 functions produce no output neither would work for me, but assuming they did produce output then it would just be down to having one of the above two features...)

(Alternatively LoadSoundEffect with it's SE_MULTIPLE parameter would do the job).

Or.... I guess you can just use LoadSound every time you want to play a sound effect to avoid the issue.
Title: Re: miniSphere 4.5.11
Post by: Rhuan on May 21, 2017, 07:47:47 pm
As an alternative answer to the above problem I tried making a sound effect object with a queue so only the number of instances needed are loaded:

Code: [Select]
function SoundEffect(filename)
{
  this.queue = [LoadSound(filename)];
  this.name = filename;
}

SoundEffect.prototype.play = function()
{
  if(this.queue[this.queue.length-1].isPlaying()==false)
  {
    this.queue.unshift(this.queue.pop());
  //  this.queue[0].stop();
  }
  else
  {
    this.queue.unshift(LoadSound(this.name));
  }
  this.queue[0].play();
}


The version there works on Sphere 1.5, to make it work on miniSphere required the the line I've commented out above - it seems that on miniSphere .play() does not work on a sound object twice without .stop() in between even if .isPlaying() is returning false.
Title: Re: miniSphere 4.5.11
Post by: Fat Cerberus on May 21, 2017, 07:54:12 pm
Hm, you're right, LoadSoundEffect() is listed in the API documentation included with Sphere 1.5.  I guess I just didn't implement it because I never found a v1 game that used it.  Good to know, I'll add it to my to-do list.

As for Audialis: Mixers not having an "isPlaying" function is by design.  The mixer is never "not producing sound" - in effect, it's like a pipeline.  It's just that sometimes it's "producing" silence (when you haven't fed it any sounds to play lately).  So there's no way to determine whether it's currently active or not because it's always active.

As for why you're not getting any sound, sound.play() (without any parameters) doesn't do what you think it does (admittedly, the wording here is somewhat ambiguous, I'll see if I can improve it):
Code: [Select]

Sound#play([mixer]);

    Begins or resumes sound playback.  When called with no `mixer` argument,
    Sound#play() will resume playback for a paused sound.  Otherwise, the sound
    is started from the beginning on the specified mixer.


If you don't provide a mixer, it's just a command to "unpause" - and if you've never played it before, the engine doesn't know what mixer to play it on.  You must explicitly specify a mixer.  Note that a default CD-quality mixer is provided for your convenience, so you don't always have to create one manually:
Code: (javascript) [Select]

var sound = new Sound('sounds/test.wav');
sound.play(Mixer.Default);
Title: Re: miniSphere 4.5.11
Post by: Rhuan on May 21, 2017, 07:59:26 pm
Hmm - using Mixer.default works, but when I tried creating a Mixer it didn't work it just gave no output.

Any thoughts on the other point about v1 (requiring the .stop() command)

Also is there a way to have the same sound playing multiple times with v2 without something like the sound queue I showed above?
Title: Re: miniSphere 4.5.11
Post by: Fat Cerberus on May 21, 2017, 08:09:44 pm

Hmm - using Mixer.default works, but when I tried creating a Mixer it didn't work it just gave no output.


Not surprising - Sphere v1 Sound#play() does exactly that.  Not sure why custom mixers don't work, something to figure out later I guess.

Quote

Any thoughts on the other point about v1 (requiring the .stop() command)


This is kind of odd, but I know there are bugs (seemingly related to Allegro's handling of streams) where the same sound fired off in rapid succession won't play every time.  .stop() forcibly drains the stream, so that's why it "fixes" the problem--but you might find it introduces little mini-stutters if you do that too much.  I've been trying to look into this bug for the past couple months, but I still have no leads as to what causes it or how to fix it. :(

Quote

Also is there a way to have the same sound playing multiple times with v2 without something like the sound queue I showed above?


Not currently, but I'm glad you brought this up - it reminds me that I wanted to introduce a "Sample" API that would serve the same purpose as Sphere v1 SoundEffect.  It's actually not good to use the Sound class for sound effects, because the sound is always streamed from disk.  That introduces undesirable latency that would be avoided if there were a way to load sounds into memory.  So yeah, there's no native way to do this now, but there should be soon. :)
Title: Re: miniSphere 4.5.11
Post by: Fat Cerberus on May 23, 2017, 11:41:05 am
@Rhuan: I'm working now on implementing both the Sphere v1 SoundEffect API as well as an enhanced Sphere v2 version that allows using Mixers.  Now that I know Sphere 1.5 supports SoundEffects, I pretty much have no choice but to implement it.  I'm sure you're not the only one that's used it in the past :)

As for why creating a mixer didn't work for you, are you sure you got the frequency right?  The frequency argument is in Hz (e.g. 44100), NOT kHz (e.g. 44), so I'm wondering if that's what the issue was.  The default mixer, for example is:
Code: (javascript) [Select]

// 44.1kHz, 16-bit, stereo, i.e. CD quality
Mixer.Default = new Mixer(44100, 16, 2);


Now that I look at the documentation again, it seems that I didn't specify frequency was in Hz.  Shame on me!
Title: Re: miniSphere 4.5.11
Post by: Rhuan on May 23, 2017, 12:45:10 pm
I was using 44.1 as my input rather than 44100 so that's probably it, will test later.

A sound effect interface with Mixers is something I'm looking forward to seeing - would like to be able to have an in game volume control menu with sound effects and music separate - which I assume this will enable.

One other query at the same time:

Sphere v1 var sound = LoadSound("blah.ogg");

Sphere v2 var sound = new Sound("sounds/blah.ogg")

Why does v2 not default to the sounds folder?
Title: Re: miniSphere 4.5.11
Post by: Fat Cerberus on May 23, 2017, 12:57:35 pm

One other query at the same time:

Sphere v1 var sound = LoadSound("blah.ogg");

Sphere v2 var sound = new Sound("sounds/blah.ogg")

Why does v2 not default to the sounds folder?


That's by design and is part of the SphereFS standard:
https://github.com/fatcerberus/minisphere/blob/master/docs/sphere2-api.txt#L89-L91

In short, paths are now standardized so that any given filename always refers to the same file regardless of which function you pass it to (except for v1 functions which maintain the backward-compatible behavior).  It's part of my initiative of making Sphere v2 more low-level and generalized, I didn't want to enforce a specific folder layout.

Quote

would like to be able to have an in game volume control menu with sound effects and music separate - which I assume this will enable.


Yep - that's the exact use case that mixers were designed for.
Title: Re: miniSphere 4.5.11
Post by: Fat Cerberus on May 24, 2017, 02:33:48 am
I'm doing some prototyping for the new Sample API, it's not ready for primetime yet (memory leaks galore!) but so far I can do this:

Code: (javascript) [Select]

let sample = new Sample('sounds/munch.wav');
while (Sphere.run()) {
key = Keyboard.Default.getKey();
if (key == Key.Space)
sample.play(Mixer.Default);
}


new Sample(...) loads the sound into memory, avoiding the overhead and latency of streaming.  And each sample.play() plays a new instance of the sound.
Title: Re: miniSphere 4.5.11
Post by: Eggbertx on May 24, 2017, 11:59:44 am
Does this address the issue I brought up about polyphony?
Title: Re: miniSphere 4.5.11
Post by: Fat Cerberus on May 24, 2017, 01:18:47 pm
If by polyphony you mean playing the same sound multiple times simultaneously, then yes.  This feature is honestly long overdue and I don't know why I didn't implement it before.
Title: Re: miniSphere 4.5.11
Post by: Eggbertx on May 24, 2017, 03:15:57 pm
Yes, that's what I meant because doing
Code: (javascript) [Select]

while(someCondition)
    (new Sound("file.ogg")).play();

isn't very effective :P
Title: Re: miniSphere 4.5.11
Post by: Rhuan on May 24, 2017, 03:28:49 pm

Yes, that's what I meant because doing
Code: (javascript) [Select]

while(someCondition)
    (new Sound("file.ogg")).play();

isn't very effective :P
Hence my sound queue...

Code: [Select]
function SoundEffect(filename)
{
  this.queue = [LoadSound(filename)];
  this.name = filename;
}

SoundEffect.prototype.play = function()
{
  if(this.queue[this.queue.length-1].isPlaying()==false)
  {
    this.queue.unshift(this.queue.pop());
    this.queue[0].stop();
  }
  else
  {
    this.queue.unshift(LoadSound(this.name));
  }
  this.queue[0].play();
}
Title: Re: miniSphere 4.5.11
Post by: Fat Cerberus on May 27, 2017, 10:56:55 am
The latest GitHub master implements the Sphere 1.5 SoundEffect API.  SE_SINGLE is not yet implemented (it's treated as SE_MULTIPLE regardless of which constant you pass in), but otherwise all the functions should be present.  (SE_SINGLE isn't that difficult to implement, but requires some internal refactoring, so I held off on including it for now.)
Title: Re: miniSphere 4.5.11
Post by: Fat Cerberus on May 28, 2017, 12:51:41 am
Update to above: SE_SINGLE sound effect mode is now implemented.

The only part of the v1 SoundEffect API I didn't implement is CreateSoundEffect(), mainly because it wasn't clear to me even after reading the Sphere 1.x source, exactly what the byte array is expected to contain.  Raw waveform data?  The verbatim contents of an audio file?  Some other format?  Sphere just seems to pass the byte array data to Audiere as-is, and I don't know enough about how Audiere works internally to divine what happens from there.
Title: Re: miniSphere 4.5.11
Post by: Rhuan on May 29, 2017, 09:59:48 am
Good work.

Also - I've been doing some more fiddling/testing, the surface blend modes don't seem to work - generally you get the same resultant image whatever you set before drawing.

(I've been trying to lighten an image and then draw it using surface_object.blitMaskSurface(surface, x, y, mask [, mask_blend_mode]) and the ADD drawing mode - but it gives me the same result as the BLEND drawing mode i.e. can only make the image darker, giving it 255,255,255 returns the original image, anything else darkens it.

I looked into your code and my best guess is that al_draw_tinted_bitmap ignores blend modes and a different function is needed.

I've been able to script a work around in sphere by setting the blend mode for the surface to ADD then drawing a rectangle over the top of the image - this gives the expected effect.
Title: Re: miniSphere 4.5.11
Post by: Fat Cerberus on May 29, 2017, 10:45:28 am
Thanks, that looks like a bug.  Sphere 1.x uses the blend mode passed in as a parameter:
https://github.com/sphere-group/sphere/blob/master/sphere/source/engine/script.cpp#L12878-L12882

Wheres miniSphere incorrectly uses the blend mode set using .setBlendMode() instead:
https://github.com/fatcerberus/minisphere/blob/v4.5.11/src/minisphere/vanilla.c#L4277-L4280
Title: Re: miniSphere 4.5.11
Post by: Rhuan on May 29, 2017, 11:40:08 am

Thanks, that looks like a bug.  Sphere 1.x uses the blend mode passed in as a parameter:
https://github.com/sphere-group/sphere/blob/master/sphere/source/engine/script.cpp#L12878-L12882

Wheres miniSphere incorrectly uses the blend mode set using .setBlendMode() instead:
https://github.com/fatcerberus/minisphere/blob/v4.5.11/src/minisphere/vanilla.c#L4277-L4280
I tested that and that wasn't a sufficient fix - adding in setBlendMode(ADD) doesn't fix the issue.

I then read about al_draw_tinted_bitmap on the allegro wiki and it sounds like it ignores blend modes and always multiplies the colours.
Title: Re: miniSphere 4.5.11
Post by: Fat Cerberus on May 29, 2017, 12:10:57 pm
Ah, I see, it specifies how to combine the mask color with the bitmap, whereas the surface blend mode specifies how the final image is combined with the surface contents.  I think I didn't implement that because, as you say, Allegeo doesn't have a way to do it.  It might be possible to do it with shaders, not sure.

I posted a feature request on the Allegro repository:
https://github.com/liballeg/allegro5/issues/768

So we'll see what happens.
Title: Re: miniSphere 4.5.11
Post by: Rhuan on May 29, 2017, 12:33:14 pm
You can get the same effect by setting the blend mode to Add then drawing both the image and a rectangle with the colour you wanted as your mask onto the surface; so assumedly you could put that into the source though I (or any other game developer using sphere) can obviously do it in the JS - just depends if you're shooting for 1-1 with sphere 1.5.
Title: Re: miniSphere 4.5.11
Post by: DaVince on June 03, 2017, 12:30:31 pm
Coming back to MiniSphere for the first time in a while, I find that the PPA does not want to function any longer. I'm running Ubuntu 17.04 and getting the following error when trying to install MiniSphere:

Code: [Select]
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:

The following packages have unmet dependencies:
minisphere : Depends: liballegro-acodec5.0 (>= 2:5.0.6) but it is not installable
              Depends: liballegro-audio5.0 (>= 2:5.0.6) but it is not installable
              Depends: liballegro-dialog5.0 (>= 2:5.0.9) but it is not installable
              Depends: liballegro-image5.0 (>= 2:5.0.6) but it is not installable
              Depends: liballegro5.0 (>= 2:5.0.7) but it is not installable
E: Unable to correct problems, you have held broken packages.


Edit: seems to be the case because Ubuntu 17.04 offers Allegro 5.2 only. I'll just compile from source for now.
Title: Re: miniSphere 4.5.11
Post by: Fat Cerberus on June 03, 2017, 12:39:22 pm
Yep, known issue, the last supported Ubuntu version is currently Xenial: https://github.com/fatcerberus/minisphere/issues/139

The Allegro packages include the version number in their package IDs, so the upgrade to 5.2 broke the dependencies.  And Yakkety stupidly removed the 5.0 packages.  Supporting 5.2-only means I give up support for 16.04 and earlier, which since that's the current LTS release, isn't really an option.  So I have to maintain two separate builds for Linux, one for Allegro 5.0 and one for 5.2, and I've been too lazy to do it.  I need to get on that ASAP. :)

What you can do for now is install Allegro 5.2 PPA (ppa:allegro/5.2) and build from source (git checkout v4.5.11 if you want stable), then the usual "sudo make install" to install it.
Title: Re: miniSphere 4.5.11
Post by: DaVince on June 03, 2017, 12:49:01 pm
That's exactly what I did. Running just fine. Maybe you should add a little note to your release, though... ;)
Title: Re: miniSphere 4.5.11
Post by: DaVince on June 03, 2017, 01:24:22 pm
I encountered another unimplemented legacy Sphere function: GradientLine(). One of the old Fallacious Gear demos seems to be using it for its battle system.

Edit: casiotone's Final Fantasy V engine also quits when starting a battle, with the error message "scripts/Battle.js:437 RangeError: area of effect extends past image (0,0,400,20)".

Also, just a random thought, but how feasible would it be to have MIDI support using fluidsynth + perhaps even custom loaded soundfonts? That would be kind of neat, but maybe a little complex?
Title: Re: miniSphere 4.5.11
Post by: Fat Cerberus on June 03, 2017, 01:53:30 pm
GradientLine should be easy, I already implement it as part of the prim module:
https://github.com/fatcerberus/minisphere/blob/master/assets/system/modules/prim.js#L79-L98
So it's just a matter of adapting that code to C.

Where's the FFV engine?  I couldn't find it in the Downloads drive.  Based on the error message, it seems to be related to either Surface#applyColorFX() or .applyLookup(), so it's possible that Sphere 1.x clamps the coordinates you give it rather than checking them strictly (which in hindsight, seems to be the norm for v1 APIs).  I'll double-check the correct behavior and make it compatible.

MIDI support is possible, would be a bit more work than the formats currently supported because for those I can just tell Allegro to load the file.  Here I would have to decode the file myself.  Now that all the mp3 patents have expired and the format is public domain, it might be a good time to look into implementing that too (although the lack of non-[L]GPL decoders may get in the way of that).
Title: Re: miniSphere 4.5.11
Post by: Fat Cerberus on June 03, 2017, 03:03:58 pm
Huh, so what I found out from looking at Sphere 1.x code is that applyLookup() throws a hard error if the rectangle extends outside the image (miniSphere does the same), HOWEVER applyColorFX does some kind of clamping instead:
https://github.com/sphere-group/sphere/blob/master/sphere/source/common/Image32.cpp#L315-L338

I don't really like that clamping behavior because it shifts the gradient, but I have to emulate it if I want full compatibility. :-\
Title: Re: miniSphere 4.5.11
Post by: DaVince on June 03, 2017, 04:11:16 pm
Quote
Where's the FFV engine?  I couldn't find it in the Downloads drive.

This just reminds me I have SO many more Sphere games to upload. I think I'll just dump them all in there somewhere later.

Quote
MIDI support is possible, would be a bit more work than the formats currently supported because for those I can just tell Allegro to load the file.  Here I would have to decode the file myself.  Now that all the mp3 patents have expired and the format is public domain, it might be a good time to look into implementing that too (although the lack of non-[L]GPL decoders may get in the way of that).

MP3 support is a good idea, especially for compatibility reasons. :) As for MIDI, I was just entertaining the thought of it (also for compatibility reasons like some of Radnen and FJ's old games), but frankly it could be a very intruiging addition when music could literally be the MIDI files and an accompanying soundfont to make it all unique for a specific game and stuff. I dunno. But like you said, it'd take more to implement it.

Edit: by the way, a LOT of old Sphere games didn't *quite* seem to want to work under MiniSphere. I'll definitely upload the whole batch of them later so you can really see what's going on and where.
Title: Re: miniSphere 4.5.11
Post by: DaVince on June 03, 2017, 04:59:45 pm
FFV engine uploaded: https://drive.google.com/file/d/0Bw-4UFVty4u1T1pvRlRpbHVocGM/view?usp=sharing
Title: Re: miniSphere 4.5.11
Post by: DaVince on June 03, 2017, 05:37:17 pm
Got a bug that is regularly acting up: some games crash with:
Code: [Select]
minisphere: /build/allegro5-3qTrvn/allegro5-5.2.2/addons/memfile/memfile.c:175: al_open_memfile: Assertion `size > 0' failed.


For example, this game (but also a bunch of others): http://vincent.tengudev.com/temp/MoonBlade.zip
Title: Re: miniSphere 4.5.11
Post by: Fat Cerberus on June 03, 2017, 05:42:23 pm
Thanks, I'll look into it once I get off work.  I'm in full-throttle development mode today :D
Title: Re: miniSphere 4.5.11
Post by: Rhuan on June 03, 2017, 08:15:25 pm
I was curious about the error DaVince pointed out.

I worked my way through the JS to try and find the trigger(s):

The cause of the hard crash was apparently using OpenFile with a file name ending in ".lng" - removing the ".lng" from the document name and at the same time removing it from the function call resulted in the crash no longer occurring.

However there was another crash - in the start of the game function there is the following:
Code: [Select]
for (var i = 0; i < game.arguments.length; i++)
{
    if (game.arguments[i] == "-d") debug = true;
}

In minisphere game.arguments is not defined and hence this causes a JS error (not a miniSphere hard crash though)

after commenting out this and changing the name above the game starts without crashing but it doesn't seem to do anything - it loads a screen with some moving clouds but doesn't then seem to take any input - not sure if this is intentional or another incompatibility I haven't found.

Edit: I've also spent a little time looking at miniSphere source to try and work out why the hard crash above occurs, I can't work it out for sure, as I don't quite follow the logic (my C skills are rather limited) but it looks to me like the function kev_open line 16 and on of Kevfile.c defines a variable slurp_size but never gives it a value - this variable is then supplied to al_open_memfile as a parameter if a certain condition is met - I cannot work out the logic of the condition - but at a guess it's if sfs_fslurp line 462 of spherefs.c fails to read the file, I can't see what the ".lng" extension has to do with this but; hopefully that's a useful start.

EDIT 2: I was wrong with the ".lng" extension point - the issue was not that at all; instead the issue was that the file was 0 bytes - changing the name somehow made it 1 byte instead of 0; the issue is also fixed if you open the file with a text editor and add some content of any kind to it.
Title: Re: miniSphere 4.5.11
Post by: DaVince on June 03, 2017, 08:40:35 pm
Quote
after commenting out this and changing the name above the game starts without crashing but it doesn't seem to do anything - it loads a screen with some moving clouds but doesn't then seem to take any input - not sure if this is intentional or another incompatibility I haven't found.

This is intentional - it's an old project that only takes some debug input (keys 1, 2 and 3). :)
Title: Re: miniSphere 4.5.11
Post by: Fat Cerberus on June 03, 2017, 10:01:52 pm
The crash seems to be caused by Allegro not wanting to open a zero-byte buffer as a memfile.  That seems like a bug to me--I'll report it.
Title: Re: miniSphere 4.5.11
Post by: Fat Cerberus on June 04, 2017, 12:24:04 am
@Rhuan If you're talking about this check:
https://github.com/fatcerberus/minisphere/blob/master/src/minisphere/kevfile.c#L25

That code is fine.  sfs_fslurp() reads an entire file in one go and returns a pointer to the first byte.  If the function succeeds, then the pointer returned is non-null and is therefore truthy, so we enter the if-clause and continue as normal (if it returns a null pointer indicating failure, the else-clause is entered instead, which is an error path).  Note that a pointer to slurp_size is passed to the slurp function--that gets set to the size of the file.  Thus slurp_size is set correctly by the time it's actually needed.  The problem here actually has to do with when slurp_size is zero--because the file was totally empty--then al_open_memfile() doesn't cooperate and crashes.

edit: For whatever reason I don't get the crashes on Windows.  It seems like maybe the Linux binaries for Allegro were compiled with assertions enabled for who knows what reason (this was stupid, since it hurts performance).  So I guess it's just the assertion failing and Allegro can actually open zero-size memfiles just fine.  MoonBlade runs without error for me on Win10 at least.
Title: Re: miniSphere 4.5.11
Post by: Rhuan on June 04, 2017, 04:36:47 am

The problem here actually has to do with when slurp_size is zero--because the file was totally empty--then al_open_memfile() doesn't cooperate and crashes.

edit: For whatever reason I don't get the crashes on Windows.  It seems like maybe the Linux binaries for Allegro were compiled with assertions enabled for who knows what reason (this was stupid, since it hurts performance).  So I guess it's just the assertion failing and Allegro can actually open zero-size memfiles just fine.  MoonBlade runs without error for me on Win10 at least.
I picked up what the issue actually was in my second look at it - see the edit2 to my post above - the stuff about slurp_size not being set was my incorrect first guess.

Seems I have asserts enabled in my Mac build too - not actually sure how to disable them - I'll look into that next time I build it.
Title: Re: miniSphere 4.5.11
Post by: Flying Jester on June 05, 2017, 03:36:43 am
The problem here actually has to do with when slurp_size is zero--because the file was totally empty--then al_open_memfile() doesn't cooperate and crashes.


This is largely the same as how memory mapping files directly with OS APIs works. A zero-length file is not obvious from a file that doesn't yet exist if you try to memory map a file without checking if it exists first.
Title: Re: miniSphere 4.5.11
Post by: DaVince on June 05, 2017, 05:40:50 am
Does miniSphere support gif files? I just found a Sphere game that uses some gif files (namely, the Sully demo) and miniSphere refuses to load it ("Error: cannot load image").

Edit: also, I'm not sure if sound.setPosition() actually works. I have a little test game that will set the playback position to the playback position + 20 if a key is held. In the original engine, this would speed up the music, but in miniSphere it just adds a little bit of noise to the sound and nothing else happens. (This is the case for both ogg and it files.) See attached file.
Title: Re: miniSphere 4.5.11
Post by: Fat Cerberus on June 05, 2017, 10:29:59 am

Does miniSphere support gif files? I just found a Sphere game that uses some gif files (namely, the Sully demo) and miniSphere refuses to load it ("Error: cannot load image").


Odd, Sully worked fine for me last time I tested it.  That was one of the games I fixed back in the day because it had map layering bugs.  So if gifs are broken, that's a regression.  But I can't see why that would have happened, unless the file is corrupt...

Quote

Edit: also, I'm not sure if sound.setPosition() actually works. I have a little test game that will set the playback position to the playback position + 20 if a key is held. In the original engine, this would speed up the music, but in miniSphere it just adds a little bit of noise to the sound and nothing else happens. (This is the case for both ogg and it files.) See attached file.


setPosition() in miniSphere is in microseconds:
https://github.com/fatcerberus/minisphere/blob/master/src/minisphere/vanilla.c#L4115

Adding +20μs to the position of course isn't going to move the playback position much (in fact it may not move at all, if it's less than the seek threshold the underlying decoder supports).  Sphere 1.x by contrast, uses some unknown unit for seeking that I never bothered trying to decipher.  It's a larger unit than microseconds, but smaller than milliseconds.  And it's not counting samples either.  So I was never able to emulate it fully.
Title: Re: miniSphere 4.5.11
Post by: Fat Cerberus on June 05, 2017, 10:53:27 am
I've verified that the reason the FFV engine crashes is due to a behavior discrepancy in Surface#applyColorFX4 compared to Sphere 1.x.  In 1.x, the coordinates you pass in are clamped to the size of the surface, and the matrices are interpolated using the unclamped values.  I just throw an error instead.  So that will need to be changed.

I haven't gotten around to testing any other old games yet, but I'd like to test at least some of them before releasing 4.6 so I can boast of greatly increased backward compatibility in the new release. :D
Title: Re: miniSphere 4.5.11
Post by: DaVince on June 05, 2017, 01:34:48 pm
Quote
Odd, Sully worked fine for me last time I tested it.  That was one of the games I fixed back in the day because it had map layering bugs.  So if gifs are broken, that's a regression.  But I can't see why that would have happened, unless the file is corrupt...

Just noticed - Sully works fine here too. I once tried to modify the game so it shows the intro and stuff, and it has to load different files for that, and that is what is failing. To be exact, it fails trying to show images/intro/retro.gif - a file that wasn't used in the original Sully Sphere demo.
This modified version of Sully can be found in that giant 1GB download with all the games I have, by the way. (Under games/DaVince to be exact.) :)

As for the microseconds thing, that makes sense! It feels a little silly to have to input a value of 10000+ before a noticeable change happens in the music, but it works.
Title: Re: miniSphere 4.5.11
Post by: Fat Cerberus on June 05, 2017, 02:46:01 pm
Yeah, that's why Sphere v2 uses floating point (in seconds) for the position.  That way you control the granularity, not the engine.
Title: Re: miniSphere 4.5.11
Post by: Fat Cerberus on June 07, 2017, 01:21:59 am

Just noticed - Sully works fine here too. I once tried to modify the game so it shows the intro and stuff, and it has to load different files for that, and that is what is failing. To be exact, it fails trying to show images/intro/retro.gif - a file that wasn't used in the original Sully Sphere demo.


It seems I was wrong and Allegro doesn't officially support GIF:
http://liballeg.org/a5docs/5.2.2/image.html#al_init_image_addon

I think earlier Windows builds used to support it because that was when I was compiling Allegro myself, and I configured it to use GDIplus to keep the .exe size down.  Windows itself of course supports GIF, so everything worked under that configuration.  On Unix platforms though (and now Windows, using the official Allegro binaries), Allegro uses libpng and libjpeg instead, so no GIF support to speak of.  This is a compatibility issue for sure, but at the end of the day I'm not convinced it's worth going out of my way to fix, since it's a lot of work and few 1.x games seem to use GIF images anyway.
Title: Re: miniSphere 4.5.11
Post by: DaVince on June 07, 2017, 02:01:42 am
Yeah, that makes sense. I don't foresee any future use happening either, so I'll just convert the files then. Thanks for looking into it.
Title: Re: miniSphere 4.5.11
Post by: Fat Cerberus on June 07, 2017, 02:31:10 am
By the way, this is neither here nor there, but GetVersion() in miniSphere returns 2.  It's the true Sphere 2.0 ;)
Title: Re: miniSphere 4.5.11
Post by: DaVince on June 11, 2017, 03:21:04 am
While going through the downloads repo, I found this spec about info.txt (https://drive.google.com/open?id=1Do7G32BuopyVVlvKMXvWEjSOOpd4dFM-cgIbkzTROzE) that I wrote back when I wanted to archive every single game onto it, for later use.

But I got to thinking... No one really uses that file, and we have the s2gm file format now. Stuff like the release date, version and contact/forum links are still useful, however, so I wondered if it wasn't a good idea to extend the s2gm format to include these, as well as having an easy interface to enter this data (support in Sphere Studio, Cell). Any thoughts on this?
Title: Re: miniSphere 4.5.11
Post by: DaVince on June 11, 2017, 03:33:19 am
Also (and this might have gotten lost somewhere in the thread), is there a way to run Cell to generate an empty project for me? Something like cell --new foldername.

Edit: okay, so I found the template in the source code (minisphere/assets/template). I assume there's a way to still have this generated through the command line, but what is it?
Title: Re: miniSphere 4.5.11
Post by: Fat Cerberus on June 11, 2017, 09:09:34 am
The template is only used by Sphere Studio right now.  Majin is meant to be the command line tool to generate projects, but I haven't started working on it yet.  That'll be one of the new features for 5.0 :)
Title: Re: miniSphere 4.5.11
Post by: DaVince on June 11, 2017, 09:14:24 am
Ahh, okay! It seems like the most useful thing. Any reason why doing something like cell --new wouldn't be sufficient? I guess you'll be going all out with what it can do?

As long as in the end you can do majin projectname and it will (or optionally won't) ask you a few basic questions about the project, I'm happy. Heck, I'm happy that running cell is a matter of just... running cell.

In the meantime, I'm writing a whole new "Getting Started" article for the wiki and just linking to a basic template project. Would this Cell.mjs work?

Code: (javascript) [Select]
/* Basic game template */

import { transpile } from 'compiler';

//In the future, this line will change into:
//Object.assign(Sphere.Game, {
describe("Game Name", {
version: 2,
saveID: 'unique-identifier-for-saving-the-game',
author: "Your name here",
summary: "Summary of the game",
resolution: '1280x720',
main: 'src/main.js',
logPath: '~/log.txt',

othervariables: 'any value you would like',
});

transpile('@/scripts/', files('src/*.mjs'));
transpile('@/scripts/', files('src/*.js'));

install('@/images',      files('images/*.*', true));
install('@/music',       files('music/*.*', true));
install('@/spritesets',  files('spritesets/*.rss', true));
install('@/sounds',      files('sounds/*.*', true));
install('@/maps',        files('maps/*.rmp', true));
install('@/maps',        files('maps/*.rts', true));
install('@/windowstyles',files('windowstyles/*.rws', true));
install('@/',            files('icon.png'));
install('@/',            files('README.txt'));


edit (fatcerberus): Add JavaScript formatting and remove extra curly braces from code
Title: Re: miniSphere 4.5.11
Post by: Fat Cerberus on June 11, 2017, 09:39:42 am
Yep, that template should work.  Just remove the extra curly braces first - the reason they are there is because I use .NET String.Format() to fill in the values, which uses {0} etc. to indicate placeholders, so you need to double them to get a literal curly brace.

As for Majin, the other idea I had for that tool was for it to be used to upgrade existing Sphere v1 projects, besides making new ones from scratch.  But maybe that's overkill these days... I'll think about it.  It would definitely be less work to integrate the functionality into Cell.
Title: Re: miniSphere 4.5.11
Post by: DaVince on June 11, 2017, 10:35:00 am
Thanks. I tweaked it a little more (a string was being terminated and I changed the folder and file inclusion a little). I'll be putting up the new wiki article pretty soon too.
Title: Re: miniSphere 4.5.11
Post by: Eggbertx on June 11, 2017, 12:57:59 pm
and we have the s2gm file format now


Really? I've never heard of that, and Radnen's Sphere Studio doesn't seem to use it.
Title: Re: miniSphere 4.5.11
Post by: Fat Cerberus on June 11, 2017, 01:23:10 pm

and we have the s2gm file format now


Really? I've never heard of that, and Radnen's Sphere Studio doesn't seem to use it.


game.json.  I originally used an .s2gm extension but changed it to standard JSON.
Title: Re: miniSphere 4.5.11
Post by: DaVince on June 12, 2017, 01:41:05 am
Yeah, I got a little confused while looking up stuff, but it's a json now. A little more difficult to associate specifically to miniSphere, but I can understand why it was done.
Title: Re: miniSphere 4.5.11
Post by: Fat Cerberus on June 12, 2017, 02:02:06 am

But I got to thinking... No one really uses that file, and we have the s2gm file format now. Stuff like the release date, version and contact/forum links are still useful, however, so I wondered if it wasn't a good idea to extend the s2gm format to include these, as well as having an easy interface to enter this data (support in Sphere Studio, Cell). Any thoughts on this?


The nice thing about a JSON-based format is that you can enter any arbitrary properties into it and the engine will just ignore anything it doesn't understand.  Of course, game.json generation being automated (via Cell) complicates matters.  I'll give this some thought--yet another thing to think about for miniSphere 5.0.
Title: Re: miniSphere 4.5.11
Post by: DaVince on June 12, 2017, 02:06:09 am
I definitely agree with the json thing. Giving it its own extension just makes it a little easier to associate and recognize, I guess.
Title: Re: miniSphere 4.5.11
Post by: Fat Cerberus on June 12, 2017, 02:58:06 am
Huh, so 4.5.11 was released on April 27.  I'll try to finalize 4.6 by the 27th, then it'll be two months since the last update.
Title: Re: miniSphere 4.5.11
Post by: Fat Cerberus on June 12, 2017, 07:53:41 pm
@DaVince: In your Cellscript template, The files() for transpile should be recursive, as most people, myself included, like to organize scripts into subfolders.  For a native Sphere v2 game that doesn't need backward compatibility, I also recommend transpiling into @/bin (rather than @/scripts), as a hint to a person looking at the distro that the game has been compiled in some way and they should look for the real source code if they want to make edits.

In other words:
Code: (javascript) [Select]

transpile('@/bin/', files('src/*.mjs', true));
transpile('@/bin/', files('src/*.js', true));
Title: Re: miniSphere 4.5.11
Post by: DaVince on June 13, 2017, 11:49:03 pm
Okay, I'll  make the changes. And now I know what the true statement is for. :P
Title: Re: miniSphere 4.5.11
Post by: Fat Cerberus on June 14, 2017, 12:05:50 am

And now I know what the true statement is for. :P


It's documented, no excuses. ;)
https://github.com/fatcerberus/minisphere/blob/v4.5.11/docs/cellscript-api.txt#L94-L103

:P
Title: Re: miniSphere 4.5.11
Post by: Fat Cerberus on June 17, 2017, 02:23:45 am
I want to refactor the standard library for miniSphere 5.0.  The code is all over the place currently and different modules use different styles; they sometimes don't even use module.exports consistently. What I think I'd like to do is reorganize it so that the CommonJS view is split at the object level--each require() module would correspond to a single namespace, function, or class--while the ES6 view would have it all merged into a single composite module, making everything more intuitive for both environments.

Node style (CommonJS):
Code: (javascript) [Select]
const from     = require('from'),
      Delegate = require('delegate'),
      Music    = require('music'),
      Scene    = require('scene');


ES6 style:
Code: (javascript) [Select]
import { from, Delegate, Music, Scene } from 'miniRT';


Which really shows the strength of the ES6 module syntax compared to old-style require() :)
Title: Re: miniSphere 4.5.11
Post by: Fat Cerberus on June 17, 2017, 10:50:51 am
There will be some minor compatibility breakage in miniSphere 4.6 in preparation for the above change.  I can still technically do that because the Sphere v2 API hasn't been finalized yet ;)
Title: Re: miniSphere 4.5.11
Post by: Fat Cerberus on June 18, 2017, 09:55:33 am
Idea I had: Example project could take advantage of classes:

Code: (javascript) [Select]
import Thread from 'thread';

class Game extends Thread
{
    constructor() {
        super();
        // startup code
    }

    on_update() { ... }
    on_render() { ... }
}

new Game();


The Thread constructor sets up the dispatch automatically, avoiding the need to deal with the Dispatch API or threader directly.
Title: Re: miniSphere 4.5.11
Post by: Fat Cerberus on June 19, 2017, 12:13:44 am
This is what the new game template looks like now:
https://github.com/fatcerberus/minisphere/blob/master/assets/template/src/main.mjs

Much cleaner and doesn't require you to understand the Dispatch API :D
Title: Re: miniSphere 4.5.11
Post by: DaVince on June 19, 2017, 03:44:01 am
Looks great, much easier to get started with!
Title: Re: miniSphere 4.5.11
Post by: Fat Cerberus on June 19, 2017, 09:32:54 am
Regarding file association, I think Linux has regex-based associations, so we could define a standard that the first field in the JSON file must be version and then use a regex like...

Code: (regexp) [Select]
^{\s*(?:\r?\n)*\s*version:\s*\d*


...to associate them with miniSphere without pulling in all JSON files.  Unfortunately Windows associates only based on file extension :(
Title: Re: miniSphere 4.5.11
Post by: DaVince on June 19, 2017, 09:43:06 am
Well, why not give it its own extension then? I still think s2gm is a good candidate (for a file extension), and the contents can just stay the JSON format that they're in.
Title: Re: miniSphere 4.5.11
Post by: Fat Cerberus on June 19, 2017, 09:49:05 am
The reason I chose game.json was for consistency with Node.js package.json.  I wanted more veteran JS developers (like myself ;)) to find Sphere approachable and familiar too, not just newbies.  Plus .json is more likely to have "Edit" as an option in its right-click menu in a default Windows setup, whereas an unknown format won't be associated with anything.

And you can always compile to an SPK ;)

I guess both have their pros and cons.  I'll revisit the issue after I finish up the module reorganization and release miniSphere 4.6.
Title: Re: miniSphere 4.5.11
Post by: DaVince on June 19, 2017, 10:01:57 am
Sounds fair, although to me it's felt like the json file just tends to be the resulting file after compiling with Cell (even if not strictly necessary), especially now that ES6 transpilation is going to be a common thing. I always found having a project file to double-click in order to run it more convenient.

Well, I guess since I've only recently started learning the New Way of Doing Things, I'm still totally used to having the sgm. :P
Title: Re: miniSphere 4.5.11
Post by: Fat Cerberus on June 20, 2017, 09:54:39 am
miniRT is being promoted to a first-class part of the Sphere v2 API.  This will split the API into two levels: the low-level Core API which can't be implemented in script such as Galileo, Audialis and the like, and sphere-runtime, which is more high-level with stuff like the Thread class, Scenario, the Music manager, and more.

I'm hoping this change will make Sphere v2 more approachable, as I've noticed that people tend to get scared away due to the more low-level design.  I try to point people to miniRT as consolation, but I think there's a stigma there due to it seeming like a third-party add-on, albeit one that happens to be bundled.  Making it an official part of Sphere v2 increases the burden for standardization, but should avoid the "inferiority complex". :P
Title: Re: miniSphere 4.5.11
Post by: DaVince on June 20, 2017, 03:11:01 pm
I think this is a really good idea, and it's good that you're pretty much immediately showcasing what it can do in the example project.
Title: Re: miniSphere 4.5.11
Post by: Eggbertx on June 20, 2017, 04:26:38 pm
I agree, especially considering image/surface blitting can be very tedious without the respective miniRT module. It's almost necessary.
Title: Re: miniSphere 4.5.11
Post by: Fat Cerberus on June 21, 2017, 12:19:56 pm
I'm considering making the Game class an official part of the Sphere v2 API.  Currently Sphere v2 has no intuitive way to pass options to the game on startup, other than adding an API that the game would have to access manually (like Sphere.Game.arguments or something), because the main script is run directly--there's no game() function like in 1.x.

See:
https://github.com/fatcerberus/minisphere/issues/166

This would avoid the need to do new Game().start() in every game's main script--the engine would do that automatically.
Title: Re: miniSphere 4.5.11
Post by: Fat Cerberus on June 22, 2017, 02:24:11 am
Just implemented the change described above.  The template now looks like this:
https://github.com/fatcerberus/minisphere/blob/master/assets/template/src/main.mjs

Exporting a Game class is optional, but methinks it will make for much easier-to-understand tutorials.  The alternative is having module-level code with global variables, Dispatch API calls (or Thread.create) and a flipscreen loop, which makes the whole thing a lot more difficult to introduce to the uninitiated.  Now you just have to introduce the concept of a class and the rest will be self-explanatory.

Overall, miniSphere 4.6 is shaping up to be a much bigger release than I originally planned.  It's going to end up being pretty close to the "definitive Sphere v2 experience", I think.  Which is good: the sooner I can stabilize the API once and for all, the better.
Title: Re: miniSphere 4.5.11
Post by: Fat Cerberus on June 26, 2017, 11:03:43 am
I'm considering retiring the Ubuntu PPA for miniSphere in 4.6 and switching to snap packages instead:
https://www.ubuntu.com/desktop/snappy

The PPA is a nightmare to maintain.  Every Linux release requires me to build at least 3 times: Once normally with make, then once each for x64 and x86 with pbuilder to ensure everything will go smoothly under Ubuntu's build sandbox.  If I forget to test with pbuilder and upload, and the build fails, Launchpad won't allow me to re-upload the package without bumping the version number.  And now it's about to get even more involved (5 builds), because I'll need to maintain separate packages for Ubuntu 16.10+ thanks to Allegro 5.2 changing the package names.  It's just a complete mess, and I don't feel supporting the most popular Linux distribution should be this hard.

Windows releases are easy: Build everything in MSVC (once!), then run Inno Setup to build the installer, test it, and upload it to GitHub.  Snap, from what I'm seeing, looks to bring that same painless workflow to Linux packaging.

Thoughts on making such a change?
Title: Re: miniSphere 4.5.11
Post by: DaVince on June 26, 2017, 02:34:52 pm
If building is a nightmare and just takes up precious time you could be using to be productive instead, I'm all for this. I'm going to have to get used to Snap packages though. :P

Quote
I don't feel supporting the most popular Linux distribution should be this hard.

I completely agree with this. You'd think it would be easier, considering there are so many people who have to deal with it on a daily basis...
Title: Re: miniSphere 4.5.11
Post by: Fat Cerberus on June 28, 2017, 10:53:00 am
It seems like making and uploading Snap packages will be easy, so that's no problem.  What's holding up the release is that I found out I'm missing most of the surface blitting functions (surface.zoomBlitSurface() et al.) so I'd prefer to implement those first.

For those curious exactly what's missing compared to Sphere 1.5, here's the list of unimplemented legacy functions in miniSphere 4.5.11:
* BezierCurve
* Filled/Outlined/GradientComplex
* CreateSpriteset
* ExecuteZoneScript
* GradientLine
* Get/SetLayerAngle
* SetLayerScaleFactorX/Y
* LoadSoundEffect + all soundeffect methods
* surface.bezierCurve
* surface.gradientLine
* surface.lineSeries
* surface.zoomBlitSurface, transformBlitMaskSurface, etc.

I've narrowed that list down a bit, notably SoundEffect is now fully supported.  The blitting functions are big though, so even though it's not a showstopper for 4.6, I want to do them anyway.

On a sidenote, I still don't understand the purpose of the Complex primitive, though I have seen a few games (notably some of Radnen's) that use it...
Title: Re: miniSphere 4.5.11
Post by: Fat Cerberus on June 28, 2017, 12:00:36 pm
So apparently Sphere 1.x implements Complex in OpenGL by drawing it pixel-by-pixel using GL_POINTS.  *shudder*  Even with a modern GPU I can't imagine that being very efficient.  And in any case it won't work properly with miniSphere's fullscreen scaling method.  I'll have to see if I can come up with a different approach.
Title: Re: miniSphere 4.5.11
Post by: Rhuan on June 28, 2017, 01:25:39 pm
I'm trying to have a think through what the API is missing vs the "dream" API.

The key things I can think of at the moment are:

- brevity, on a per function or per interface basis sphere v2 methods are often longer/more flexible than wanted in common cases, I guess miniRT is meant to help with this, but sphere v1 functions may run faster for some things at that point, consideration point rather than any specific actions at the moment.

- fast graphics editing abilities as all images and surfaces are pushed to the graphics card if you want to edit them it's slower than you might like
Title: Re: miniSphere 4.5.11
Post by: Fat Cerberus on June 28, 2017, 01:47:12 pm
The Core API is low-level by design-and usually more performant when used properly because it more closely matches how things work at the hardware level, particularly wrt Galileo.  That's why miniRT exists, to provide a more high-level abstraction for common use cases-and is generally the preferred API for user-level code.  Hence why I've decided to promote miniRT to an official part of Sphere v2 :)
Title: Re: miniSphere 4.5.11
Post by: Flying Jester on June 28, 2017, 03:59:35 pm

So apparently Sphere 1.x implements Complex in OpenGL by drawing it pixel-by-pixel using GL_POINTS.  *shudder*  Even with a modern GPU I can't imagine that being very efficient.  And in any case it won't work properly with miniSphere's fullscreen scaling method.  I'll have to see if I can come up with a different approach.


I can't imagine that the complex primitive is very important, though. I never even bothered implementing it, even in the faster GL driver I wrote for Sphere 1.6, and I never had any issues.
Title: Re: miniSphere 4.5.11
Post by: Fat Cerberus on June 28, 2017, 04:08:08 pm
Yeah, that's why I haven't implemented it until now either.  Radnen used it a lot in his games but otherwise it didn't seem to get much use.
Title: Re: miniSphere 4.5.11
Post by: Flying Jester on June 28, 2017, 06:34:13 pm
I also wouldn't look too much at how the old Sphere drivers worked, they tended to be assuming a very fill-limited architecture, rather than vertex-limited. A much smarter solution would just have been to do per-scanline fills using GL_LINES. There was a reason that I decided to write my own, although doing so strengthened my resolve that an API with persistent primitives was needed for performance that was significantly higher than a software solution.

That was a part of what lead my first prototypes of Galileo, and I determined that you got a HUGE boost in performance by forcing triangle-based geometry definitions (slow geometry is hard to specify, fast geometry is simpler), and by encouraging persistent geometry (all modern graphics frameworks use this in some manner). The design also lends itself to "accumulator" based asynchronicity, as it was implemented in Sapphire, which vastly improves performance.
Title: Re: miniSphere 4.5.11
Post by: Fat Cerberus on June 29, 2017, 10:12:00 am
So it turns out that there are a few bugs caused by miniSphere's fullscreen implementation:
https://github.com/fatcerberus/minisphere/issues/167
https://github.com/fatcerberus/minisphere/issues/170

This affects Galileo, which is unfortunate.  It will need to be fixed, but I'm not sure if I'll do it in 4.6 or hold off until 5.0.  Anyway, the gist is that, because miniSphere applies a scaling transformation to everything drawn to the backbuffer while in fullscreen, primitives like ShapeType.Lines don't work correctly.  The lines end up "thinner than a pixel" because they are drawn at the native resolution and only the vertices get scaled.

Not sure of a good fix yet.  My idea was to use my own backbuffer without scaling applied and draw that to the "real" backbuffer as a textured quad, but that could be a performance issue?  I'm not sure.
Title: Re: miniSphere 4.5.11
Post by: Rhuan on June 29, 2017, 01:40:41 pm
I had noticed a few inexplicable graphical anomalies when using minisphere, not sure why I hadn't mentioned them before

In the attached you can see a screenshot of text drawn with the system font over the system windowstyle drawn with a colour mask - there's an odd white line on the window, it goes away in windowed mode but is there in full screen - is this related to the issue you're talking about?

(The red circles behind are something I've been messing around with brownian motion spinning rings as a background they're "fun"...)
Title: Re: miniSphere 4.5.11
Post by: Fat Cerberus on June 29, 2017, 01:56:18 pm
That's indeed related--and hopefully goes away with the same fix--but no, it's not the same issue.  The bug in your case is due to an interaction between fullscreen scaling and font atlasing.  How miniSphere's font renderer works is to load all font glyphs into a single, large atlas texture.  When using an integral scale factor like 1x or 2x, you can accurately address individual texels.  When using a non-integer scale, however, as often happens in fullscreen, the GPU starts interpolating texels and sometimes you end up pulling in stuff from adjacent glyphs.  Hence the weird artifacts.

Incidentally, tilesets and spritesets are loaded the same way, so the map engine suffers from this bug too.
Title: Re: miniSphere 4.5.11
Post by: Rhuan on June 29, 2017, 02:24:59 pm
I use zoombBlitted single pixel width shapes in the project I've been working on and then generally display quite well in miniSphere.

I also use a surface that I draw graphic primitives (including rectangles with a single pixel outline) and images to and then zoomBlit and it displays well full screen - so there is a way to draw and scale without problems.
Title: Re: miniSphere 4.5.11
Post by: Fat Cerberus on June 29, 2017, 02:31:21 pm
Yeah, zoomBlit draws whole images, so that's not a problem.  The GPU will simply clamp to the edges of the texture.  The artifacts in your screenshot happen because the font renderer draws individual glyphs from a single large image containing all of them (called an "atlas")--and in fullscreen where a non-integer scaling factor is used, the GPU starts sampling pixels from adjacent tiles/glyphs.
Title: Re: miniSphere 4.5.11
Post by: Flying Jester on June 29, 2017, 02:38:48 pm
I managed to avoid any of those issues by only ever using nearest-neighbor filtering for zooms. That should never cause bleedover from adjacent atlas entries.
Title: Re: miniSphere 4.5.11
Post by: Rhuan on June 29, 2017, 02:51:44 pm
Three questions:

a) could you use an array of images for a font instead of an "atlas"?

b) If zoomBlit handles single pixel width primitives on the surface it's drawing fine then is there a related way to fix your primitives rendering issue?

c) this idea of an Atlas sounds interesting, is this effectively like surface.clonesection(x,y,w,h).blit(x,y)?
Title: Re: miniSphere 4.5.11
Post by: Fat Cerberus on June 29, 2017, 02:58:22 pm
a) Atlases are used because they improve performance.  A LOT.  OpenGL is a state machine and switching textures is expensive.  Using an atlas minimizes the number of state changes that need to be done each frame, which is very important for, e.g. text and map rendering which involve many distinct images.

b) The only thing that will fix it properly, I think, is to use a game resolution-sized texture as the backbuffer and scale that up, instead of applying the scaling to rendering operations themselves.  It's just that implementing this is a large undertaking and requires refactoring the internal flipping code, so it will take me a day or so to implement.  It's not hard, just tedious.  FJ's solution would work too, but it would lead to really ugly fullscreen since everything drawn to the backbuffer is scaled up.  I still want the bilinear filtering, just without the bleedover.

c) Functionally, yes, it's the same concept, but without the added expense of creating additional throwaway images.
Title: Re: miniSphere 4.5.11
Post by: Rhuan on June 29, 2017, 03:04:21 pm
a) is there anyway to expose Atlas type functionality within the API? (though without the sampling outside of the designated region)

b) OK, well I look forward to seeing fix
Title: Re: miniSphere 4.5.11
Post by: Fat Cerberus on June 29, 2017, 03:09:42 pm
The sampling issue totally goes away with the fix I mentioned.  But yeah, you can already implement an atlas using Galileo, it's just kind of tedious.  It'd probably be worth adding an atlas module to the standard library (or enhance Prim.blit() to make things easier.  I'll see what I can do, thanks for the idea.  :D
Title: Re: miniSphere 4.5.11
Post by: Rhuan on June 29, 2017, 03:35:57 pm

The sampling issue totally goes away with the fix I mentioned.  But yeah, you can already implement an atlas using Galileo, it's just kind of tedious.  It'd probably be worth adding an atlas module to the standard library (or enhance Prim.blit() to make things easier.  I'll see what I can do, thanks for the idea.  :D
Time to learn how to use the sphere v2 graphics api, you've finally given me an incentive that has me interested :) It previously just always seemed like an unnecessary pain.

This probably also means it's time to bin my use of Sphere v1 spritesets as now I can just use a single image for a whole spriteset and hopefully get performance enhancements from doing so.
Title: Re: miniSphere 4.5.11
Post by: Fat Cerberus on June 29, 2017, 03:59:10 pm
This is a good incentive for me too, to look at the implementation of Galileo and see if the internal wiring can improved at all.  There's a bit too much internal componentization at present and it probably does more texture changes than strictly necessary, preventing it from performing as well as it could.
Title: Re: miniSphere 4.5.11
Post by: Rhuan on June 29, 2017, 04:04:19 pm
So lets build this up a few steps at a time, have I got the below right?

Situation:
- I want to make a spriteset by combining multiple input images (containing body/clothing/weapons etc)
- I then want to draw parts of that spriteset (the different frames for different scenes)

1. Load the various parts as textures

2. Create a surface to be the spriteset

3. For each texture apply it to a shape and draw the shape on the surface (could use one shape over and over?)
OR could instead use a shape per texture and make them into a model then draw the model - not sure if there's any benefit to this unless you can add and remove shapes from the model easily? (can't see this in the api)

4. Convert the surface into a texture - the Sprite

5. When I wish to draw it make a shape and texture it with the sprite setting the vertex.u and vertex.v properties in order to determine which parts of the sprite are drawn?

6. Use translation matrices to move the shape around the screen when needed

7. draw the shape and flipscreen etc when wanted (or go into proper v2 mode and have a dispatch render script that draws the shape)

8. How do I change what frame of the sprite is showing, can I edit the shape's vertices' u and v properties or do I have to bin the shape and make a new one?
Title: Re: miniSphere 4.5.11
Post by: Rhuan on June 29, 2017, 04:51:33 pm
Follow up question:

So I can't find a way to get at the u and v values of a vertex, the transformation operations allowed only seem to get the x and y unless I'm missing something?

In light of this the only ways I can see to use a texture as an atlas are to either:
a) keep creating and destroying shapes OR
b) make an array of shapes one for each possible piece of the atlas then draw the one you want

Which of these is better? Or are they both bad? Could you create a type of transformation to edit u and v i.e. slide the texture around?

The other option to achieve the same effect on screen that I can see is to have one shape that you move around and keep retexturing with an array of Textures instead of the atlas.
Title: Re: miniSphere 4.5.11
Post by: Fat Cerberus on June 29, 2017, 05:01:38 pm
Yeah, you can only transform the spatial position of a vertex, not its texture coordinates.  At least not without using a vertex shader.

Ideally you'd want to keep the Shapes around.  Recreating them all the time would be no better performance-wise than Sphere v1 blitting because you're constantly re-uploading vertices.  The idea of a Shape is that you upload vertices to the GPU once and then reuse them (as-is) over and over again using transformation matrices.  Constantly making new ones all the time defeats the purpose. :)

I can explain in-depth later; I'm currently at work so no time to get into the nitty-gritty.
Title: Re: miniSphere 4.5.11
Post by: Rhuan on June 29, 2017, 05:36:03 pm
Hmm ok I can see four ways to implement the system I'm thinking of with the Galileo API:

Option 1: A large array of Shapes textures using an atlas - have to determine which shape to draw based on certain parameters each render cycle

Option 2: One Shape per sprite that is textured using an atlas and converted into a model; I would then need to have written a glsl shader that will allow me to input variances to the u and v values of its vertices, my update code when then use Model#setInt(name, value); when needed to specify the correct u and v values that the shader would then set.
(unsure why I can't specify a shader for a shape and have to make a model out of it first) - I note I'd need to learn some glsl but I've found a tutorial already.

Option3: Creating each shape only when drawing it (using an atlas to hold the graphical data so only one texture is ever needed but continually destroying the shapes)

Option4: forget about the atlas, split the image into all of it's seperate frames so I have loads of textures, use one shape and retexture it whenever I need to change frame
Title: Re: miniSphere 4.5.11
Post by: Fat Cerberus on June 29, 2017, 06:05:59 pm
The best approach, IMO, would be to create one Shape per sprite with the corresponding texture coordinates for its position in the atlas.  You would set the shape to have its "center of gravity" (usually the center of the collision base) at (0,0) and you could then simply use a translation matrix to put it wherever you like.

Regarding using a shader when drawing individual shapes... that feature should be possible to add, actually.  The idea of a Model is to optimize drawing for multiple shapes using the same parameters, but there's no reason I couldn't optimize internally to minimize render state changes when multiple shapes using the same texture/shader are drawn individually.

Again, you don't want to continually destroy and recreate shapes--that's no better performance-wise than just blitting the old-fashioned way.  Shapes are meant to be persistent, which is why they are objects.  Keeping them around avoids unnecessary communication with the GPU--which is quite expensive.

This is why I needed someone (other than me) to try out the new graphics API, it's been very untested ground until now. :)
Title: Re: miniSphere 4.5.11
Post by: Fat Cerberus on June 29, 2017, 06:09:38 pm
To clarify a bit on the performance thing, when you go "new Shape" what that does is to upload the array of vertices you passed in to the graphics card.  Later, drawing it is just a matter of telling the graphics card "draw using this vertex buffer I gave you before".  Things like old-style blit have to upload four vertices on the fly for every blit, which adds up after a while.
Title: Re: miniSphere 4.5.11
Post by: Rhuan on June 29, 2017, 06:20:03 pm
I think you missed a point - each sprite has many frames.

E.G. if the sprite is facing North and standing still that's one frame, if it's facing north and has it's left leg forward that's another etc.

Hence my hope was that one atlas could be all the frames of that sprite, and I could move which part of the atlas the sprite's shape is textured with then:

a) I can apply transformations to move the sprite around AND
b) apply a different type of transformation to change which frame of the sprite is being drawn AND
c) only need two objects to do the whole thing (one texture and one shape)

With my option 2 above I think I can do this if I have a model as well and write some glsl.
Title: Re: miniSphere 4.5.11
Post by: Rhuan on June 29, 2017, 07:49:23 pm
Ok, I got the shader working, heading towards that option 2, would be nice if you could add shader support to shapes to remove an object from the code, but it works without that.

BUT I seem to have found a weird error when I draw a Model screen.flip() does not clear the backbuffer afterwards but FlipScreen() does.
Title: Re: miniSphere 4.5.11
Post by: Fat Cerberus on June 29, 2017, 07:56:53 pm
That's a weird bug - the only difference between the two flip functions is that FlipScreen doesn't reset the clipping rectangle for compatibility (screen.flip does) and uses SetFrameRate rather than screen.frameRate.  Everything else should be the same.  I'll look into it later, though.

Note that Model can't be completely removed from the API because you might want to use the same Shape with different shaders and/or uniform values.  I agree that the extra complexity isn't always needed, though.  I'll see what I can do to improve it going forward. :D
Title: Re: miniSphere 4.5.11
Post by: Rhuan on June 29, 2017, 08:07:34 pm
When I'm asking for shader support at the Shape level it would need the uniform setting features as well, are you saying that model is there so you can draw the same shape with two different values at the same time by having it in different models, I'd assumed that that wouldn't work, not sure why I'd assumed this, but I had.

My "custom" shader is just the default vertex shader you'd written with two uniform floats added in that get added to the x and y components of the varying_texcoord.

FYI I'm thinking of trying to build what I'm working on out into a "standard" sprite library for your v2 api so we can deprecate .rss with something good.
Title: Re: miniSphere 4.5.11
Post by: Radnen on June 29, 2017, 10:24:02 pm
Fat Cerberus, have you thought about putting "dead zones" between sprites on the atlas? If you add a 1px or 2px transparent gap, the artifacting should go away. It helped the atlases in my SSFML build, for instance.
Title: Re: miniSphere 4.5.11
Post by: Fat Cerberus on June 29, 2017, 11:43:23 pm

When I'm asking for shader support at the Shape level it would need the uniform setting features as well, are you saying that model is there so you can draw the same shape with two different values at the same time by having it in different models, I'd assumed that that wouldn't work, not sure why I'd assumed this, but I had.


If I remember correctly, FJ's original design didn't allow that, but I realized there was no technical reason why it couldn't be supported, so miniSphere does allow it.
Title: Re: miniSphere 4.5.11
Post by: DaVince on June 30, 2017, 07:34:29 am
implementing this is a large undertaking

it will take me a day or so

This is pretty great. ;D
Title: Re: miniSphere 4.5.11
Post by: Fat Cerberus on June 30, 2017, 09:12:27 am

implementing this is a large undertaking

it will take me a day or so

This is pretty great. ;D


:)
What can I say, I'm good at managing my time.
Title: Re: miniSphere 4.5.11
Post by: Fat Cerberus on June 30, 2017, 12:10:17 pm
I started experimenting with the fullscreen fix, and I found that doing it the way I want to has an additional positive side effect: Screenshots will be saved at the game's native resolution, rather than being the physical size of the window as they are in 4.5 and earlier.  That's especially nice for fullscreen screenshots, which right now come out letterboxed at 1080p :)
Title: Re: miniSphere 4.5.11
Post by: Rhuan on June 30, 2017, 04:52:05 pm
A few more thoughts from fiddling with Galileo:

1. It seems that to apply a colour mask to a shape you either have to retexture it OR I need to add more to my shader...

2. You don't have the different blend modes that v1 has, but maybe it's best to implement these via the shader - (possible project for me this evening)

3. I think it would be good if you could  pass values to the shader at the shape level, would mean you could move a model around whilst having parts of it change colour differently, rather than needing to track multiple models.

4. For code cleanliness would be great if we could pass vectors to the shader - if I do as much with shaders as I'm thinking of now I think the uniform setting is going to get quite clunky I can obviously make helper functions to make it look neater but would be nice if it was neater.

(Yes I may have just spent a while dreaming about what I can do with glsl...)
Title: Re: miniSphere 4.5.11
Post by: Fat Cerberus on June 30, 2017, 05:01:17 pm
To apply a color mask to a shape you just change the color of its vertices.  This is how Prim.blit() applies the mask you give it.  As for blend modes, you're right, that needs to be done in the pixel shader.  Allowing predefined blend modes alongside shader support seems like mixing metaphors and I don't want to make the v2 API too complicated.

Agreed on the usefulness of passing vectors.  That would be good for passing colors, for instance.
Title: Re: miniSphere 4.5.11
Post by: Rhuan on June 30, 2017, 05:07:18 pm

To apply a color mask to a shape you just change the color of its vertices.  This is how Prim.blit() applies the mask you give it.  As for blend modes, you're right, that needs to be done in the pixel shader.  Allowing predefined blend modes alongside shader support seems like mixing metaphors and I don't want to make the v2 API too complicated.

Agreed on the usefulness of passing vectors.  That would be good for passing colors, for instance.
But unless I'm missing something I can't re-colour the vertices without making a new shape?

I'm looking at how to be as efficient as possible with my use of shapes but at the moment unless I'm missing something the only writeable property of a shape is a texture and masking a texture is just a pain (convert to surface then draw a coloured shape over it then convert back)

-> Going to work on the shader method instead.

Note: whilst with a bit of effort I'm growing to like the concepts in this Galileo interface, I think it is nowhere as beginner friendly as the v1 graphics interface -> there needs to be clearer tutorials/guidance/something if we want programming beginners to be able to dive into this.
Title: Re: miniSphere 4.5.11
Post by: Fat Cerberus on June 30, 2017, 05:12:42 pm
Yeah, that's by design.  Constructing a shape is tantamount to uploading it to the graphics card.  If I allowed the properties to be changed afterwards, I think people would tend to want to do that and then wonder why their game is so slow (every modification would need to re-upload the whole shape).  Forcing the user to construct a new one makes the re-upload explicit.  If you need to dynamically change things, it's indeed recommended to do it in the shader.
Title: Re: miniSphere 4.5.11
Post by: Fat Cerberus on June 30, 2017, 05:17:43 pm

Note: whilst with a bit of effort I'm growing to like the concepts in this Galileo interface, I think it is nowhere as beginner friendly as the v1 graphics interface -> there needs to be clearer tutorials/guidance/something if we want programming beginners to be able to dive into this.


Yep, that's intentional.  Beginners are meant to use the Prim module for exactly this reason.  Galileo is the low-level foundation everything is built on, but it's definitely designed to be used as middleware, rather than the primary graphics interface for game-level code.  It'd be way too tedious to try to use it directly in game code without having a framework built on top of it.
Title: Re: miniSphere 4.5.11
Post by: Flying Jester on June 30, 2017, 06:08:28 pm
If you do use it directly, you usually want to put it into your game's engine code. In my prototype games (mostly RTS and puzzles, but the concepts apply to RPGs) I usually attached Shapes to a character [proto]type (ie a sprite), and then had a Group/Model for each character that included the shape of the type. This is more efficient than the v1 API which amounts to immediate mode, and actually works pretty well with a prototypical object model, and for an RPG type of game (each prototype of a character or enemy 'class' in the RPG sense, or unit type in an RTS has a Shape, and each instance has a Group/Model, which means the actual sprite is the Shape, and is not duplicated between individual characters in memory or on the GPU).
Title: Re: miniSphere 4.5.11
Post by: Rhuan on June 30, 2017, 06:59:30 pm
Spotted two things I think are bugs, or at least aren't in line with the api as far as I read it.

1. If I have two different models using the same shader and set a uniform value for one of them it sets it for both of them.

2. Color#red [read/write]
Color#green [read/write]
Color#blue [read/write]
Color#alpha [read/write]
Don't exist, though:
Color#r
Color#g
Color#b
Color#a
Do exist and work.
Title: Re: miniSphere 4.5.11
Post by: Fat Cerberus on June 30, 2017, 11:14:39 pm
(1) is definitely a bug, uniform values are supposed to be specific to each model.  I'll look into that.

(2) is just a documentation mistake, I've flip-flopped back and forth between full and abbreviated color components and I don't always remember to update the API doc when I do.  So thanks for the heads-up that I need to change it again. ;)

edit: The shader bug is kind of strange.  Before a model is drawn, the first thing the engine does after switching in the shader is to set its uniforms:
https://github.com/fatcerberus/minisphere/blob/v4.5.11/src/minisphere/galileo.c#L209-L222

That's done unconditionally, regardless of which shader was set before.  In fact it seems that after every model, the shader is reset to the default Allegro one (shader_use(NULL)).  The engine isn't even being clever by reusing the shader--it's reset each time.  So I don't understand why your uniform values are bleeding over between models...
Title: Re: miniSphere 4.5.11
Post by: Rhuan on July 01, 2017, 05:37:11 am
It's specifically if I have two models using the same shader if I set a uniform value for one it sets it for both, it's easy to work around I just have to set all the uniforms each time I'm going to draw anything.

I've had a look at the source in galileo.c and I could be wrong but I think what it's doing is:

looping through all uniforms set for the current model and setting them for the shader BUT the values set for the shader persist so:

A) Create model B and set for it uniforms C&D

B) draw model B -> uniforms C&D have the correct values no other uniforms are set

C) Create model F and set for it uniforms J&K

D) draw Model F -> uniforms C,D,J&K are all set, C&D with the values from model B and J&K with the values from Model B

E) draw Model B -> uniforms C,D,J&K are all set etc.

Now if each time I create a model I initialise all relevant uniforms for that model the issue would go away.

The source level options to fix it would be either:
a) add in a step of initialising all uniforms the shader has before looping through the model's uniforms
OR
b) make uniforms a shader level action rather than model level

(I think option (a) would be better)
Title: Re: miniSphere 4.5.11
Post by: Fat Cerberus on July 01, 2017, 10:13:10 am
b) is how shaders were originally implemented, but I didn't like that because you might want different models to use the same shader but with different parameters.  So I made it model-level like it is now.

I'll look into it again and do some tests.
Title: Re: miniSphere 4.5.11
Post by: Fat Cerberus on July 01, 2017, 11:07:43 am
I found it!  For each model the engine only has a list of uniforms the user has explicitly set.  So if multiple models use the same shader, and you set values for one model without setting them for the other, the other model will just skip the uniform-setting loop due to an empty list.  I think when I implemented that, I was under the impression that changing shaders would reset the uniforms already set, but it seems OpenGL maintains them.

I'll have to see if there's a way to work around it.  I can't just explicitly set all the unset uniforms to zero, because they might have default values in the shader itself.  So this is a stumper.
Title: Re: miniSphere 4.5.11
Post by: Rhuan on July 01, 2017, 11:15:25 am
Shader issue: Are you sure that the shader has default values? there's no way to set any from within minisphere (per the api) Unless you mean the 3 allegro ones that the core engine provides? You could initialise all then reset those 3 or something OR perhaps this should just be a documentation point; include in the api that if a shader contains any uniforms they've defined they may get odd results if they don't initialise them for each model that uses that shader? (Maybe add vector uniforms so not as many are needed so initialising them all for each model will be less of a flaff)

Other matter:
I've been thinking more on the beginner friendly-ness (and also the needs of the project I'm half work on).

What I don't like about prim as a bridge for beginners is it ditches all the advantages of galileo and doesn't point towards its strengths which is part of why I wasn't keen on galielo to begin with; I was looking at the way prim used it and basically thinking:
"this is just unnecessary".

So I'm thinking of putting together a game graphics engine using galileo that can handle sprites, primitives, and other things AND use persistent shapes to take advantage of the speed galieleo can give; if it works out perhaps it could be a useful supplement to the core engine.

-> As a potential follow up if this goes well I'm thinking of building a galileo powered flexible map engine using tiled maps (unless someone else is already making one?)
Title: Re: miniSphere 4.5.11
Post by: Fat Cerberus on July 01, 2017, 11:31:16 am

Shader issue: Are you sure that the shader has default values? there's no way to set any from within minisphere (per the api) Unless you mean the 3 allegro ones that the core engine provides? You could initialise all then reset those 3 or something OR perhaps this should just be a documentation point; include in the api that if a shader contains any uniforms they've defined they may get odd results if they don't initialise them for each model that uses that shader?


I just meant that if a shader does something like:
Code: (javascript) [Select]

uniform float foo = 8.12;


If the engine were to zero uninitialized uniforms (to avoid the original bug), then we introduce a new bug of the default value getting destroyed.  No go.

Quote

Other matter:
I've been thinking more on the beginner friendly-ness (and also the needs of the project I'm half work on).

What I don't like about prim as a bridge for beginners is it ditches all the advantages of galileo and doesn't point towards its strengths which is part of why I wasn't keen on galielo to begin with; I was looking at the way prim used it and basically thinking:
"this is just unnecessary".

So I'm thinking of putting together a game graphics engine using galileo that can handle sprites, primitives, and other things AND use persistent shapes to take advantage of the speed galieleo can give; if it works out perhaps it could be a useful supplement to the core engine.

-> As a potential follow up if this goes well I'm thinking of building a galileo powered flexible map engine using tiled maps (unless someone else is already making one?)


The reason for Prim to exist is not really to show off Galileo but instead to provide v1-style immediate-mode primitives, which are generally much easier to understand for beginners.  It's true that the Galileo code to support them is overcomplicated, but that's the point: modern GPUs just aren't designed to work that way (if you look at how the v1-analogue primitives are implemented in Allegro, they do the same thing the prim code does).  Galileo, as originally designed by FJ, was set up to mirror the way modern graphics hardware actually works.  So if you use it for what's designed to do, the resulting code makes much more sense. :)

As for a Galileo-powered map engine, TurboSphere's map engine actually did just that--in fact in his case the whole map was a single Shape (I experimented with doing that internally, but large
maps ended up taking a performance hit).  I would like to implement a new Galileo-powered JS map engine in miniSphere 5.0 as well.  That's why I decided to do a 4.6 release, because there are a lot of useful changes on the trunk already and I still have my work cut out for me. :P
Title: Re: miniSphere 4.5.11
Post by: Rhuan on July 01, 2017, 11:38:40 am

I just meant that if a shader does something like:
Code: (javascript) [Select]

uniform float foo = 8.12;


If the engine were to zero uninitialized uniforms (to avoid the original bug), then we introduce a new bug of the default value getting destroyed.  No go.
Doesn't the shader get executed at draw time hence re-doing that definition? Or is it that only the function is called at draw time?

Quote
I would like to implement a new Galileo-powered JS map engine in miniSphere 5.0 as well.  That's why I decided to do a 4.6 release, because there are a lot of useful changes on the trunk already and I still have my work cut out for me. :P
How about I do this bit for you so you can focus on core engine code?
Title: Re: miniSphere 4.5.11
Post by: Fat Cerberus on July 01, 2017, 11:48:58 am

Doesn't the shader get executed at draw time hence re-doing that definition? Or is it that only the function is called at draw time?


I think the default values are set up at compile time, not runtime.  The shader program itself is persistent (hence why you have a Shader object), the engine just binds and unbinds them at draw time.  Regarding your other suggestion, making it a documentation point: I would like to avoid having too many "gotchas" like that, even if they're documented, because one of the goals with the Sphere v2 API is to make Sphere code self-documenting.  If one skim overs the API and sees that uniform values are set per-model, then subsequently looks at code where one model has values set and the other doesn't, they will intuitively expect the second model to just use the default values.  So the behavior in practice should match that expectation.

The issue underlying the bug is that, in OpenGL, uniform values are specific to a shader program (and GL maintains them across rebinds), while in miniSphere multiple models can share the same program but with different values.  Like you suggested before, making uniforms shader-level would avoid the problem, however it would be less user-friendly and defeat the purpose of the Model object.
Title: Re: miniSphere 4.5.11
Post by: Rhuan on July 01, 2017, 11:56:55 am
Trying to think through how to resolve that shader issue...

Possible option:
1. When a shader is compiled read back from it what it's uniforms are
2. Store those uniforms as an array somewhere.
3. When drawing a model clone the above array
4. write the uniforms from the model to the clone
5. write the clone to the shader

Title: Re: miniSphere 4.5.11
Post by: Rhuan on July 01, 2017, 12:21:35 pm
Just thinking on how a map engine should work with galileo, what about doing it with each layer of the map being an atlas. You have one model per layer the size of the clipping window you wish to draw in (either the whole screen or a subsection as suits) and you then use your shader to varying the texture coordinates so that only the right part of the map is in frame. (rather than moving the shape and drawing excessive amounts of it that are offscreen).

(I think the default shaders should be slightly more extensive than the ones you have at the moment in order to support stuff like that), after I've polished them a little will be happy to share the ones I'm using.
Title: Re: miniSphere 4.5.11
Post by: Fat Cerberus on July 01, 2017, 12:23:49 pm
As long as you're willing to license your code under BSD/MIT, I'd be more than happy to let you develop the Sphere v2 map engine. :)
Title: Re: miniSphere 4.5.11
Post by: Fat Cerberus on July 01, 2017, 01:58:13 pm
So it turns out that thanks to the recent fullscreen fix, I can now implement BezierCurve() and it will render properly.  Yay!

The only thing I can't figure out is why BezierCurve() is documented to just plot points along the curve, rather than making a continuous line.  Seems that would limit its utility.
Title: Re: miniSphere 4.5.11
Post by: Rhuan on July 01, 2017, 07:07:10 pm
Couple more comments...

1. Keyboard.isPressed(Key.Escape) gives me "Undefined not callable" - what have I got wrong? (I'm trying to cut v1 calls from any code I write but as well as still needing FlipScreen I find I've now found I still need v1 keys)

2. Transformations, the features you've given us are fine if we want to move objects around, rotate them and zoom in and out, but what if we want to distort them? Any chance you could give us read/write access to the underlying matrices? (It may already be there but if it is I can't find it).
Title: Re: miniSphere 4.5.11
Post by: Fat Cerberus on July 02, 2017, 02:22:29 am

1. Keyboard.isPressed(Key.Escape) gives me "Undefined not callable" - what have I got wrong? (I'm trying to cut v1 calls from any code I write but as well as still needing FlipScreen I find I've now found I still need v1 keys)


Keyboard.Default.isPressed(Key.Escape)Keyboard is just the class name; you need to have a Keyboard object first to read any keys from it.  The API is set up this way to leave the option open for supporting multiple input devices in the future without a breaking change.  By the way, what's the FlipScreen bug again?  It's several pages back at this point and I don't feel like digging for it. :P

Quote

2. Transformations, the features you've given us are fine if we want to move objects around, rotate them and zoom in and out, but what if we want to distort them? Any chance you could give us read/write access to the underlying matrices? (It may already be there but if it is I can't find it).


No, I think you're right, I didn't provide any access to the underlying matrix cells in the API.  I don't think that's too hard to do, I could definitely add support for that to v4.6.  It'd probably be most intuitive to have array-style access, e.g. transform_obj[ i][j]
Title: Re: miniSphere 4.5.11
Post by: Rhuan on July 02, 2017, 04:00:22 am
Thanks for the replies, and I agree on array style access to transformations.

As for the FlipScreen issue (it was screen.flip I had an issue with hence using FlipScreen instead) but I can't manage to reproduce the issue now, so maybe it was some other mistake I'd made.
Title: Re: miniSphere 4.5.11
Post by: Fat Cerberus on July 02, 2017, 09:46:35 am
If it helps, here's a comparison of the implementations of FlipScreen() and screen.flip():
v1: https://github.com/fatcerberus/minisphere/blob/v4.5.11/src/minisphere/vanilla.c#L1851-L1856
v2: https://github.com/fatcerberus/minisphere/blob/v4.5.11/src/minisphere/pegasus.c#L1224-L1230

The only difference is the framerate variable used, which for v1 defaults to 0 (unlimited) while screen.frameRate (v2) defaults to 60, and that screen.flip() resets the clipping rectangle afterwards.  So maybe the lack of frame limiting in v1 by default was what was causing your issue?
Title: Re: miniSphere 4.5.11
Post by: Fat Cerberus on July 02, 2017, 11:54:30 am
Idea for new feature: 3D rendering, complete with projection support, camera, etc.  Sphere has always been a 2D engine, but it would be a nice bonus if we could also do basic 3D stuff with it.  Things like loading model files, etc. could be added in time.
Title: Re: miniSphere 4.5.11
Post by: DaVince on July 02, 2017, 11:55:13 am
I wonder if a maximum frame rate shouldn't be enforced even in v1, like eg. a default frame rate of 240 or so. I've managed to lock my entire system up once just by accidentally getting it stuck in an infinite loop. (Note: I don't remember how it happened, just THAT it happened.)

...Actually, scratch that. That might have been a loop without any rendering/flipping going on at all. I dunno. Just something was weird.
Title: Re: miniSphere 4.5.11
Post by: Rhuan on July 02, 2017, 12:06:13 pm

Idea for new feature: 3D rendering, complete with projection support, camera, etc.  Sphere has always been a 2D engine, but it would be a nice bonus if we could also do basic 3D stuff with it.  Things like loading model files, etc. could be added in time.
I was going to ask for 3D the other day, doesn't feel like the API needs much expansion (other than allowing us to fiddle with transformations more as already mentioned) the main thing that would need work I guess is the rendering code under the hood.

As in you can already set Z coordinates BUT if you keep them between +1 and -1 they seem to do nothing and if you go outside of that range then the shape isn't drawn.
Title: Re: miniSphere 4.5.11
Post by: Eggbertx on July 02, 2017, 04:22:52 pm
Is it currently possible for a game to set the window icon?
Title: Re: miniSphere 4.5.11
Post by: Fat Cerberus on July 02, 2017, 04:26:54 pm

As in you can already set Z coordinates BUT if you keep them between +1 and -1 they seem to do nothing and if you go outside of that range then the shape isn't drawn.


I never figured this out myself.  I guess something to do with the Z buffer?  It caused a few bugs in the past where I forgot to zero a vertex buffer in the engine and ended up with nothing getting drawn.  It took me a while to figure that one out. :P

It looks like most of the useful 3D utility functions are not in Allegro 5.0.  So I'll need to require 5.2 or later.  A non-issue for windows releases (allegro is static linked), but may be a problem for Linux.  We'll see.
Title: Re: miniSphere 4.5.11
Post by: Fat Cerberus on July 02, 2017, 04:28:13 pm

Is it currently possible for a game to set the window icon?


Not at runtime (yet); any custom icon should be 32x32 and packaged in the dist as icon.png.
Title: Re: miniSphere 4.5.11
Post by: Rhuan on July 02, 2017, 04:35:07 pm


As in you can already set Z coordinates BUT if you keep them between +1 and -1 they seem to do nothing and if you go outside of that range then the shape isn't drawn.


I never figured this out myself.  I guess something to do with the Z buffer?  It caused a few bugs in the past where I forgot to zero a vertex buffer in the engine and ended up with nothing getting drawn.  It took me a while to figure that one out. :P

It looks like most of the useful 3D utility functions are not in Allegro 5.0.  So I'll need to require 5.2 or later.  A non-issue for windows releases (allegro is static linked), but may be a problem for Linux.  We'll see.
My current build of minisphere 4.5 for mac is a) statically linked and b) already using allegro 5.2 :P so non-issue for macos support.
Title: Re: miniSphere 4.5.11
Post by: Eggbertx on July 02, 2017, 04:43:34 pm


Is it currently possible for a game to set the window icon?


Not at runtime (yet); any custom icon should be 32x32 and packaged in the dist as icon.png.

That's fine, that's what I meant.
Title: Re: miniSphere 4.5.11
Post by: Fat Cerberus on July 02, 2017, 11:36:34 pm

I wonder if a maximum frame rate shouldn't be enforced even in v1, like eg. a default frame rate of 240 or so. I've managed to lock my entire system up once just by accidentally getting it stuck in an infinite loop. (Note: I don't remember how it happened, just THAT it happened.)


As long as a script is calling FlipScreen() or DoEvents() regularly, or using the Sphere v2 event loop like in the example project, then you will always be able to close the window even if the CPU is pegged.  Of course if you have an infinite loop that doesn't do any event processing, that will cause problems, but there's not really anything miniSphere can do about that.  JS is single-threaded so it comes with the territory, unfortunately.
Title: Re: miniSphere 4.5.11
Post by: Fat Cerberus on July 03, 2017, 12:02:12 am
So I was going to fix all the spriteset bugs... but then I found out, Sphere 1.x does this for all functions that accept a spriteset parameter:
https://github.com/sphere-group/sphere/blob/master/sphere/source/engine/script.cpp#L1199-L1660

Over 450 lines of code just to read in a spriteset argument.  You see, Sphere 1.x does something very strange with spriteset parameters that it doesn't do for any other object: It reconstructs a native spriteset from the properties of the spriteset object passed in--as opposed to just having the object carry a native pointer and reading that back (like miniSphere does).  This should be a lot of fun to try to emulate.  I was originally going to fix it for miniSphere 4.6, but now I'm thinking it might be better to hold off on tackling that one.  It's not going to be fun, I can see that already.
Title: Re: miniSphere 4.5.11
Post by: Fat Cerberus on July 03, 2017, 01:35:23 am
Here's a peak at all the enhancements and fixes that'll be in miniSphere 4.6:
https://github.com/fatcerberus/minisphere/blob/master/CHANGELOG.md

And here're the release notes (listing breaking changes):
https://github.com/fatcerberus/minisphere/blob/master/RELEASES.md
Title: Re: miniSphere 4.5.11
Post by: Flying Jester on July 03, 2017, 01:39:02 am
Well, a lot of the lines in that function are because one: Sphere 1.x does all GC rooting there manually (and does a lot of unnecessary GC rooting), and two: Spritesets don't have actual constructors/classes, so it is literally doing this in the most complicated way possible.
Title: Re: miniSphere 4.5.11
Post by: Fat Cerberus on July 03, 2017, 01:52:35 am
Yeah, I noticed all the GC rooting.  It should honestly have been enough just to root the spriteset object itself and then all its properties would be guaranteed to be reachable for the duration of the call.  Even then I'm not sure any GC rooting was necessary at all.  Oh well, it's very old code, I'll give it a pass. :)

Anyway, the main point I was getting at is that Sphere 1.x reconstructs an internal spriteset object on-the-fly every time it accepts a Spriteset as an API parameter.  That's not going to be fun to replicate because the structure goes 3-4 levels deep in spots and Duktape's API is stack-based like Lua.  It can be done and it's not even really challenging, just extremely tedious.  Unfortunately, there do exist games that rely on the behavior, so I'll bite the bullet sooner or later.  Just not now. ;)
Title: Re: miniSphere 4.5.11
Post by: Rhuan on July 03, 2017, 02:27:42 am

Here's a peak at all the enhancements and fixes that'll be in miniSphere 4.6:
https://github.com/fatcerberus/minisphere/blob/master/CHANGELOG.md

And here're the release notes (listing breaking changes):
https://github.com/fatcerberus/minisphere/blob/master/RELEASES.md
Is there any chance of getting direct access to transformation matrices and setting vector uniforms added to 4.6?

I'd really appreciate it if possible.
Title: Re: miniSphere 4.5.11
Post by: Fat Cerberus on July 03, 2017, 02:37:22 am
Vector uniforms should be pretty easy, I can probably implement it in 10 minutes or so since the framework is already in place (note UNIFORM_INT/FLOAT_VEC in galileo.c).  I could maybe also add .setColor() for setting a 4-vector using the components of a Color.

Transformation matrix access I want to use array syntax, but that's a bit tricky since it's a two-level array.  I could probably come up with something easily enough, though.  If worse comes to worst I could postpone the feature for 4.6.1 since it's not a breaking change, but I'll try to avoid that.  I'm trying to finalize 4.6.0 by the Fourth of July if I can :D
Title: Re: miniSphere 4.5.11
Post by: Rhuan on July 03, 2017, 04:55:47 am
Thinking about vector uniforms I'd like to be able to do 2vectors, 3vectors and 4vectors, as there are uses for each and having the right size makes the shader more efficient.

For transformations, something like transformation#matrix[­i][j] would be great.
Title: Re: miniSphere 4.5.11
Post by: Rhuan on July 03, 2017, 03:38:45 pm
Follow up comment about Uniform setting, is there a way to have one function set a vector of varying lengths? As I feel that otherwise there simply end up being too many setUniform functions in the api relative to anything else, alternatively could it be possible to create an object or array of uniforms in some way then have one set function that you pass the object to as a parameter?
Title: Re: miniSphere 4.5.11
Post by: Fat Cerberus on July 03, 2017, 11:44:21 pm
It would in theory be possible to have one setUniform() method to rule them all, however there's a complication.  JS only has one number type: double.  This means it's impossible to distinguish a float value from an integer if one is only using the type of the value argument.  So we'd end up with something like:

Code: (javascript) [Select]

model.setUniform("floatunif", Uniform.Float, 3.0);
model.setUniform("intunif", Uniform.Int, 5);
model.setUniform("transform", Uniform.Transform, myTransform);


Which is a lot more wordy than the current API for no benefit.

Anyway, the particulars of the API can be worked out during 5.0 development.  For now, this is what I can get into miniSphere 4.6 for you (already implemented on the trunk):
https://github.com/fatcerberus/minisphere/blob/master/docs/sphere2-core-api.txt#L846-L874

edit: Aww, syntax highlighting doesn't work anymore. :(
Title: Re: miniSphere 4.5.11
Post by: Fat Cerberus on July 04, 2017, 12:16:17 pm
@Rhuan: Just implemented direct matrix access:
https://github.com/fatcerberus/minisphere/commit/5303ba45d81274b60db26e2746b679db44791c8c

I'll try to finalize everything and post 4.6.0 before the night is out.
Title: Re: miniSphere 4.5.11
Post by: Rhuan on July 04, 2017, 12:44:59 pm

@Rhuan: Just implemented direct matrix access:
https://github.com/fatcerberus/minisphere/commit/5303ba45d81274b60db26e2746b679db44791c8c

I'll try to finalize everything and post 4.6.0 before the night is out.
Good work, but, is it read only or read/write? YOu say read only then give an example of changing a value (I need write access for some of the stuff I want to do)
Title: Re: miniSphere 4.5.11
Post by: Fat Cerberus on July 04, 2017, 12:52:07 pm
Sphere.Game is documented as read-only too.  It just means that the property itself can't be reassigned, that is, you can't do:

Code: (javascript) [Select]

transf.matrix = somethingElse;


I need to come up with a better way to communicate that because as you point out, "read-only" in this context is misleading.
Title: Re: miniSphere 4.5.11
Post by: Rhuan on July 04, 2017, 01:17:56 pm

As long as you're willing to license your code under BSD/MIT, I'd be more than happy to let you develop the Sphere v2 map engine. :)

MIT or BSD license is fine, topic to discuss/plan map engine and sprite handler: http://forums.spheredev.org/index.php/topic,1466.msg9159.html
Title: Re: miniSphere 4.6.0
Post by: Fat Cerberus on July 04, 2017, 11:26:41 pm
miniSphere 4.6.0 is available!  There are way too many improvements to summarize here; just look through the changelog to find out what's new. ;D
Title: Re: miniSphere 4.6.0
Post by: Rhuan on July 05, 2017, 03:43:38 am

miniSphere 4.6.0 is available!  There are way too many improvements to summarize here; just look through the changelog to find out what's new. ;D
Good work.

I've now built it for MacOs and it runs fine, I had to repeat my two edits from before:
1. Adding the path hack to the assets path function (I don't like the sphere system folder being invisible which is the alternative to this) as well as removing the "if" you'd put around it in the engine path function
2. renaming panic to d_panic (as a MacOs system include of some kind defines something else called panic)

I think I'm going to see if I can build the other components as well (sphereRun, Cell etc), not just the main app then do a release of sorts.

Comments on changed graphical functionality
- the graphical anomalies are gone BUT
- my brownian motion spinning rings now look awful (I think they were relying on some oddity of how the full screen resizing worked to ake them look good before - they always looked bad in windowed mode)

Other note...
Be interested in your comments in my map engine topic, hoping to code the core of it in the next two weeks.
Title: Re: miniSphere 4.6.0
Post by: Fat Cerberus on July 05, 2017, 08:26:08 am

1. Adding the path hack to the assets path function (I don't like the sphere system folder being invisible which is the alternative to this) as well as removing the "if" you'd put around it in the engine path function
2. renaming panic to d_panic (as a MacOs system include of some kind defines something else called panic)


Thanks, I'll fix the panic thing on my end while it's still fresh in memory.  I had forgotten about it because it's in dyad.c, which isn't my code.  I was trying to figure out what you were referring to with that, and then I just gave up and did a Find in Files to figure out what was wrong.  It'll be fixed for good now. :)

Regarding that path hack: See, I'm kind of jealous that it's even possible for you to hide the system folder on macOS.  If I could easily make a version of the engine for Windows that was entirely self-contained, assets and all and only required distributing the engine executable with a game, I would have done it a long time ago.  So you wanting to make said folder visible again is a little backwards to my sensibilities. ;)  That said, I do understand why you want things that way and actually had posted this Allegro feature request a while ago:

https://github.com/liballeg/allegro5/issues/700

While the request was acknowledged (the bug post is tagged), it doesn't seem to be high on their list of priorities.  I would implement it myself but I don't have a Mac.
Title: Re: miniSphere 4.6.0
Post by: Fat Cerberus on July 05, 2017, 01:33:44 pm
I found out why objects drawn outside of the [-1.0,1.0] Z range don't get rendered.  Allegro's default projection is:

Code: (c) [Select]

al_orthographic_transform(&trans, 0, 0, -1.0, width, height, 1.0);



Which is to say, pixel-perfect for X/Y with the near and far clipping planes at -1.0 and 1.0, respectively.  Everything outside that range gets clipped.

I'm curious now, does Galileo honor Z order?
Title: Re: miniSphere 4.6.0
Post by: Rhuan on July 05, 2017, 01:57:11 pm

I found out why objects drawn outside of the [-1.0,1.0] Z range don't get rendered.  Allegro's default projection is:

Code: (c) [Select]

al_orthographic_transform(&trans, 0, 0, -1.0, width, height, 1.0);



Which is to say, pixel-perfect for X/Y with the near and far clipping planes at -1.0 and 1.0, respectively.  Everything outside that range gets clipped.

I'm curious now, does Galileo honor Z order?
I'm just thinking, it's probably possible to setup 3d drawing within the transformation api within sphere, now you've given full matrix access, can write all the properties to what we need them to be to project 3d onto 2d, unless allegro is going to clip it anyway?
Title: Re: miniSphere 4.6.0
Post by: Fat Cerberus on July 05, 2017, 02:02:59 pm
Allegro isn't clipping it, OpenGL does--because Allegro told it to use an orthographic projection with the near and far planes at -1.0 and 1.0.  That's not hardcoded, it's just a default.

But yeah, we're most of the way there already, it's just missing an API to actually apply the projection matrix.  Setting model_obj.transform just affects the modelview matrix, I don't think you can apply projections at that stage in the pipeline.
Title: Re: miniSphere 4.6.0
Post by: Rhuan on July 05, 2017, 02:14:59 pm
I'm thinking more on the path-hack for macos.

I've had a read through your code and there are two relevent places that the assets_path function are called:

1. Finding the system directory: - for a version of sphere intended to be shipped with games having this hidden away would be fine. (though less helpful for a game developer)

2. Finding the games folder: - if Sphere is intended to be shipped with more than one game in a v1 style this is a problem

I had been thinking if it was just point 1 I could make Sphererun as a mac unix-style executable that can ignore the whole issue and be used for development and then minisphere as a deployment bundle with the system directory hidden in it, but as long as v1 is a thing (and to be honest it's more of a thing than v2 at the moment) that's not going to work.
Title: Re: miniSphere 4.6.0
Post by: Fat Cerberus on July 05, 2017, 02:22:38 pm
GetGameList() actually uses engine_path(), which has the path-hack already implemented, it's just "if (false)"'d out because I didn't know how to detect whether the engine is bundled or not without having to resort to writing ObjC code.  Going three steps uplevel breaks everything in anything other than a bundled OSX app :)
Title: Re: miniSphere 4.6.0
Post by: Rhuan on July 05, 2017, 03:03:09 pm

GetGameList() actually uses engine_path(), which has the path-hack already implemented, it's just "if (false)"'d out because I didn't know how to detect whether the engine is bundled or not without having to resort to writing ObjC code.  Going three steps uplevel breaks everything in anything other than a bundled OSX app :)
Check line 1872 of vanilla.c (ExecuteGame function) that uses assets path for the games directory, so may need to change that.

The startup game is also looked for inside assets_path so would need to hide that away as well as the system folder.

On the bundle detection point:
- We don't need to detect a bundle at run time - you make the pick when you make your xcode Target (effectively a makefile) people aren't normally going to take the executable out of a bundle at a later point and shouldn't expect it to work if they do - if you do that to any other mac application it normally fails to open.
- seems to me that the correct condition is something like this:
#if defined __apple__ && !defined MINISPHERE_SPHERUN

On a different note sphererun is working nicely, just had to make a new xcode target and a tweak a few settings. I guess I should build ssj next?
Title: Re: miniSphere 4.6.0
Post by: Fat Cerberus on July 05, 2017, 03:06:44 pm
I assume __apple__ is not defined for a unix-style app?  If that's the case, that would work perfectly. :D
Title: Re: miniSphere 4.6.0
Post by: Rhuan on July 05, 2017, 03:22:50 pm

I assume __apple__ is not defined for a unix-style app?  If that's the case, that would work perfectly. :D
No __apple__ is defined for a unix style app it is defined for anything you build to run on a mac BUT you wouldn't make the release version as a unix style app; nothing intended for a "normal" end user is built as a unix style app it's just not done hence my suggesiton of the two conditions 1. __apple__ and 2. not MINISPHERE_SPHERUN

Unrelated point can't get SSJ to run, assume I've messed something up building it:
Sphererun output:
waiting for connection from debug client...
connected to debug client at 127.0.0.1
Abort trap: 6

SSJ output:
./ssj -a
SSJ 4.6.0 Sphere JavaScript debugger (x64)
a symbolic JavaScript debugger for miniSphere
(c) 2015-2017 Fat Cerberus

connecting to 127.0.0.1:1208... OK.
verifying... error!
Title: Re: miniSphere 4.6.0
Post by: Fat Cerberus on July 05, 2017, 03:31:10 pm
Hmm, what happens if you do "ssj game_dir" on the command line (ssj and spherun must be built in same directory)?
Title: Re: miniSphere 4.6.0
Post by: Fat Cerberus on July 05, 2017, 03:54:08 pm
Google indicates abort trap 6 is a segfault, so I'm assuming some initialization is failing for whatever reason and the engine does a null pointer access.  Could you maybe run spherun under the Xcode debugger and then attach SSJ from the command line to see where it crashes?
Title: Re: miniSphere 4.6.0
Post by: Fat Cerberus on July 06, 2017, 02:48:25 am
Just implemented basic support for setting projection matrices:
https://github.com/fatcerberus/minisphere/commit/590d68a2176acbf60921e800d302ec08aba07e98

Currently projection is only supported for things drawn to the screen, but I'd like to implement it for surfaces as well.  First I need to work out the API for project3D(), though.  Right now it works like glFrustum() which makes it more flexible, but maybe a bit too advanced for beginners.
Title: Re: miniSphere 4.6.0
Post by: Rhuan on July 06, 2017, 03:17:46 am

Google indicates abort trap 6 is a segfault, so I'm assuming some initialization is failing for whatever reason and the engine does a null pointer access.  Could you maybe run spherun under the Xcode debugger and then attach SSJ from the command line to see where it crashes?
Trying that and doesn't get me anything useful, running through the xcode debugger instead of aborting sphererun just hangs, no error, but nothing happening either, I'll have to put in some extra printfs or something to try and track it through, may be able to do that later today, not sure.
Title: Re: miniSphere 4.6.0
Post by: Fat Cerberus on July 06, 2017, 03:31:54 am
By the way if you decide you want to try building Cell at some point in the future, you can compile Spectacles with it as a test:
https://github.com/fatcerberus/spectacles-i

kh2Bar works too:
https://github.com/fatcerberus/kh2bar
Title: Re: miniSphere 4.6.0
Post by: Rhuan on July 06, 2017, 01:30:51 pm
My plan was to build all of the components of miniSphere for 4.6 then do a Mac release of sorts but seems that only the main app is working correctly at the moment.

Will look at it more later today or tomorrow.

Title: Re: miniSphere 4.6.0
Post by: FBnil on July 06, 2017, 01:43:57 pm
I really got to begin making time for my game... sigh. Anyways, thanks F.C. for continuing support.
Title: Re: miniSphere 4.6.0
Post by: Rhuan on July 06, 2017, 02:35:47 pm
I was missing how the xcode debugger works, it was giving me an output, not sure how useful it is, the function where it's falling over is duk_debugger_detach with the following dump:

sphereRun`duk_debugger_detach:
    0x1000062fa <+0>:  pushq  %rbp
    0x1000062fb <+1>:  movq   %rsp, %rbp
    0x1000062fe <+4>:  leaq   0x2504dd(%rip), %rsi      ; "duk_api_debug.c"
    0x100006305 <+11>: leaq   0x2504e6(%rip), %rcx      ; "no debugger support"
    0x10000630c <+18>: movl   $0x60000f2, %edx          ; imm = 0x60000F2
    0x100006311 <+23>: callq  0x100002a8c               ; duk_err_handle_error at duk_error_macros.c:21
Title: Re: miniSphere 4.6.0
Post by: Fat Cerberus on July 06, 2017, 02:40:02 pm
That IS useful.  Duktape looks to be throwing a "no debugger support" error when miniSphere attempts to attach the debugger.  This indicates you might have compiled spherun without defining MINISPHERE_SPHERUN which causes the other options not to be set properly for a debugger-enabled build.
Title: Re: miniSphere 4.6.0
Post by: Rhuan on July 06, 2017, 02:54:20 pm

That IS useful.  Duktape looks to be throwing a "no debugger support" error when miniSphere attempts to attach the debugger.  This indicates you might have compiled spherun without defining MINISPHERE_SPHERUN which causes the other options not to be set properly for a debugger-enabled build.
I definitely compiled with MINISPHERE_SPHERUN set, and have just triple checked this.
Title: Re: miniSphere 4.6.0
Post by: Fat Cerberus on July 06, 2017, 03:12:09 pm
OH!  You also have to set DUK_OPT_HAVE_CUSTOM_H I think.  I've made that mistake before, too (you'll want to define that for the normal build too or glitches may happen)
Title: Re: miniSphere 4.6.0
Post by: Rhuan on July 06, 2017, 03:29:15 pm

OH!  You also have to set DUK_OPT_HAVE_CUSTOM_H I think.  I've made that mistake before, too (you'll want to define that for the normal build too or glitches may happen)
Fixed :)
Title: Re: miniSphere 4.6.0
Post by: Fat Cerberus on July 07, 2017, 02:35:59 am
Surfaces now support setting a transform, allowing projections to be used in Render-to-Texture operations:
https://github.com/fatcerberus/minisphere/blob/master/docs/sphere2-core-api.txt#L894-L903

Now I'll have to give it a workout by getting it to draw a cube. :D

The implementation could probably be optimized better, but it gets the job done and works as designed.  The specific issue is that the engine explicitly sets the render target for every drawing operation.  That probably isn't a big deal in practice for most 2D games but won't scale well at all for more advanced 3D.  From what I've seen Allegro tries to lighten the load a bit by avoiding state changes if they can be avoided (reusing shaders and whatnot), but the way the engine is set up, it also explicitly swaps out the modelview and projection matrices every time, which entails a lot of uploading matrices to the GPU.  That kind of thing can't be good for performance. :P

edit: @Rhuan: By the way, when you say "fixed", do you mean that SSJ works now...?
Title: Re: miniSphere 4.6.0
Post by: Rhuan on July 07, 2017, 02:56:38 am

Surfaces now support setting a transform, allowing projections to be used in Render-to-Texture operations:
https://github.com/fatcerberus/minisphere/blob/master/docs/sphere2-core-api.txt#L894-L903

Now I'll have to give it a workout by getting it to draw a cube. :D

The implementation could probably be optimized better, but it gets the job done and works as designed.  The specific issue is that the engine explicitly sets the render target for every drawing operation.  That probably isn't a big deal in practice for most 2D games but won't scale well at all for more advanced 3D.  From what I've seen Allegro tries to lighten the load a bit by avoiding state changes if they can be avoided (reusing shaders and whatnot), but the way the engine is set up, it also explicitly swaps out the modelview and projection matrices every time, which entails a lot of uploading matrices to the GPU.  That kind of thing can't be good for performance. :P

Sounds like good work, well done.

On the performance point, if you're redrawing something surely there's a good chance it's moved and hence you need to change a matrix?

Quote
edit: @Rhuan: By the way, when you say "fixed", do you mean that SSJ works now...?
Yes that is what I meant sorry for being unclear.
Title: Re: miniSphere 4.6.0
Post by: Fat Cerberus on July 07, 2017, 09:20:14 am
The modelview matrix (Shape#transform) of course changes for every object drawn, but there's no need to set the projection matrix and render target every time.  In most cases those are going to be the same throughout a frame.
Title: Re: miniSphere 4.6.0
Post by: Fat Cerberus on July 09, 2017, 01:39:19 am
After the feature-packed release of miniSphere 4.6, I'm taking a break from adding more features to do some internal refactoring.  A lot of the original code doesn't match current naming conventions and coding style, and internal objects don't always align with their "official" names in the API, making them harder to find at a glance.  So I'm taking the time to get things up to speed and hopefully make the code a bit easier to maintain going forward.
Title: Re: miniSphere 4.6.0
Post by: Fat Cerberus on July 11, 2017, 03:15:20 pm
The Socket API is getting an overhaul.  Instead of a Socket object there will be Client and Server objects.  Server works as before, but Client is reusable and you'll be able to connect it to a different host at any time, or reconnect it if it gets disconnected.  It's kind of annoying that you can't currently reuse a socket that got disconnected.  I know that's normally how BSD sockets work, but Sphere should be at a bit higher level than that.
Title: Re: miniSphere 4.6.0
Post by: Rhuan on July 11, 2017, 05:11:26 pm
I was browsing your github and saw https://github.com/fatcerberus/minisphere/issues/153

Are you still thinking of replacing duktape? How big a job would it be?
Title: Re: miniSphere 4.6.0
Post by: Fat Cerberus on July 11, 2017, 05:20:41 pm
That's a big reason for the recent code refactorings, actually -- I'm trying to decouple the actual implementation of the engine parts from the JS hooks so that swapping out JS engines in the future is just a matter of rewriting the hooks.  If you look at most of the Sphere v1 implementation in vanilla.c, the JS functions do a lot of actual work themselves and they shouldn't.
Title: Re: miniSphere 4.6.0
Post by: Rhuan on July 11, 2017, 05:56:07 pm
What would the benefits be of swapping the JS engine, is it just about access to newer features or is it about speed or some combination?

(I haven't run into any real speed problems in miniSphere apart from:
a) the galileo ones you've fixed for your next release and
b) when I wrote a stupid piece of code that called array.sort on an array with 50 elements 50 times per frame - and yes I mean per frame not per second...).

Largely unrelated point - what's your current planned release date for v5? I'd like to have the sprite/map/collisions pieces I'm working on ready before v5 but I've been super busy at work recently and am going to be going away for a bit soon so will likely not be able to get them finished until late august at the earliest.
Title: Re: miniSphere 4.6.0
Post by: Fat Cerberus on July 11, 2017, 06:00:13 pm
I have quite a backlog of stuff to implement for 5.0 so it won't be for a while.  See the v5.0.0 tag in github issues for the full list.

As for changing the JS engine, a big benefit would be support for ES6+ syntax without transpiling.  That would lighten the learning curve for beginners because they wouldn't have to learn to use Cell right away.  Also it would likely be much faster for CPU-bound tasks.  You could get more done between frames, for example.
Title: Re: miniSphere 4.6.0
Post by: Fat Cerberus on July 12, 2017, 02:50:49 am
Starting with miniSphere 4.7, the engine will require a shader-capable GPU to start.  Even Intel HD Graphics has supported shaders and the like for years now and about the only thing that doesn't are crappy XP-era Intel chipsets.  Those are veritable dinosaurs these days and if someone still wants to make games for them, well, Sphere 1.5 is still floating around. ;)

Mainly, my motivation is to get rid of all the untested code to support older hardware.  For example if a shader-ready GL context can't be created then the engine disables shaders, creates a fixed-function context and uses slightly different code paths for rendering.  Or if vertex buffers aren't supported, the vertices are kept in a memory buffer instead.  Pretty much all modern machines support these features so it just adds up to a lot of legacy-compatibility code that never gets tested.  Better just to remove it and require a modern machine for a modern engine.
Title: Re: miniSphere 4.6.0
Post by: Rhuan on July 12, 2017, 03:24:58 am

Starting with miniSphere 4.7, the engine will require a shader-capable GPU to start.  Even Intel HD Graphics has supported shaders and the like for years now and about the only thing that doesn't are crappy XP-era Intel chipsets.  Those are veritable dinosaurs these days and if someone still wants to make games for them, well, Sphere 1.5 is still floating around. ;)

Mainly, my motivation is to get rid of all the untested code to support older hardware.  For example if a shader-ready GL context can't be created then the engine disables shaders, creates a fixed-function context and uses slightly different code paths for rendering.  Or if vertex buffers aren't supported, the vertices are kept in a memory buffer instead.  Pretty much all modern machines support these features so it just adds up to a lot of legacy-compatibility code that never gets tested.  Better just to remove it and require a modern machine for a modern engine.
Makes sense - particularly in light of the things you can do with Shaders and Galileo - running in no-shader mode would make the output significantly different for any of my Galileo using code, and assuming we get other people into using this stuff, likely for there's too - so supporting it is just a bad idea.
Title: Re: miniSphere 4.6.0
Post by: Fat Cerberus on July 12, 2017, 12:46:34 pm
I'm going to add a legacy support module like legacy.c or something to the source code.  In the process of refactoring the internal sockets code, I realized that the Sphere v1 sockets implementation depends on some "magic" behavior of the socket object (i.e. a server socket magically transforms into client upon first connection), which I then had to work around with special cases in the v2 implementation where that behavior is undesirable.  Removing the magic behavior from the internal socket object of course breaks the v1 API.  I think it would be better to separate legacy support code into its own module so it can't hold back the implementation of new stuff anymore.

Oh, I implemented clipping for surfaces.  In 4.7 you'll be able to call surface.clipTo(x, y, w, h) to restrict drawing to the specified region of the surface.
Title: Re: miniSphere 4.6.0
Post by: Flying Jester on July 12, 2017, 10:20:48 pm
There's a reason that I had proposed for Pegasus (and implemented in TurboSphere) clients and servers explicitly being different (a Socket and a Listener).
Title: Re: miniSphere 4.6.0
Post by: Fat Cerberus on July 12, 2017, 11:46:51 pm

There's a reason that I had proposed for Pegasus (and implemented in TurboSphere) clients and servers explicitly being different (a Socket and a Listener).


Yep, I did the same in Sphere v2 (Server and Socket in my case).  it's just that because I fully support the legacy API too, I ended up implementing "magic sockets" as a feature of the internal socket object, as a special case (because it would have been even more complicated to implement it at the JS interface layer).  Since the "magic" behavior can easily be implemented on top of a more traditional setup though, it'll be easy to refactor it out into its own legacy section of the codebase.
Title: Re: miniSphere 4.6.0
Post by: Eggbertx on July 13, 2017, 04:26:08 am
As a suggestion, would it be possible to give the option of not repeatedly adding the currently held key/button to the queue, relying on some kind of event system for getting key/button presses and releases instead?
Title: Re: miniSphere 4.6.0
Post by: Eggbertx on July 13, 2017, 05:02:37 am
For example, as it is now, doing something like
Code: [Select]

const term = require('term');
const kb = Keyboard.Default;
var current_key;
function game() {
   threads.create({
      update: function() {
         current_key = kb.getKey();
         return true;
      }
   );
}


Everything seems to work fine, but when I open a terminal, because getKey() clears the queue, I'm unable to enter. The terminal is an example that would/might benefit from a proper built-in input event system.
Title: Re: miniSphere 4.6.0
Post by: Fat Cerberus on July 13, 2017, 12:43:26 pm
Ideally the console should be stealing input focus from the rest of the game, but I don't have any way of implementing that right now.  I have a GitHub issue open to implement something along those lines, but I haven't done anything for it yet:
https://github.com/fatcerberus/minisphere/issues/161

With the low-level way the Core API is designed, technically the only thing the engine should provide is kb.isPressed()... but I decided to have getKey() as well because that allows games to take input for typing and at the system's usual repeat rate.  Otherwise the game would have to implement a key queue itself and typing would feel totally different from other applications.
Title: Re: miniSphere 4.6.0
Post by: Rhuan on July 13, 2017, 05:34:25 pm
Have you had a chance to look into the frame rate limit bug (shown by turning on the speed limit in my graphics test script)?
Title: Re: miniSphere 4.6.0
Post by: Fat Cerberus on July 13, 2017, 05:38:01 pm
Not yet, I've been trying to get a lot of the refactoring I wanted to do out of the way before I get too close to the 4.7 release.  Once I get close to releasing I don't like to make major architectural changes, so I'm getting it done now.

I'll definitely look at it soon, it shouldn't be losing half the frames when the unlimited frame rate is just over 60fps.
Title: Re: miniSphere 4.6.0
Post by: Eggbertx on July 13, 2017, 06:07:32 pm

Ideally the console should be stealing input focus from the rest of the game, but I don't have any way of implementing that right now.

That's what I thought it was supposed to do and it's why I was confused at first. As far as how to implement it, I think I mentioned earlier in the thread that it might be a better idea to have it built-in. Or do what some other games do and have it completely pause execution aside from the terminal. That's what I was going for when I made my own terminal (which I've since lost) and I never had any issues with it, unless you entered in bad JavaScript code, since it was designed to directly handle code.


Otherwise the game would have to implement a key queue itself and typing would feel totally different from other applications.

I thought most APIs/frameworks use events like SDL_PollEvent/SDL_KEYDOWN/SDL_KEYUP, JavaScript DOM events, etc. That was what I had in mind here.

Edit: also Dispatch.onRender draws over the terminal as well.
Title: Re: miniSphere 4.6.0
Post by: Fat Cerberus on July 13, 2017, 09:46:26 pm

Edit: also Dispatch.onRender draws over the terminal as well.


Which miniSphere version?  The console has infinite priority so it should always render last.  I know I did fix a bug related to incorrect Dispatch order in 4.6, maybe that's what you're encountering:
https://github.com/fatcerberus/minisphere/commit/f88b4aa96f975d4c11030abf2e02071f25adf7d0
Title: Re: miniSphere 4.6.0
Post by: Fat Cerberus on July 13, 2017, 11:08:29 pm

Or do what some other games do and have it completely pause execution aside from the terminal. That's what I was going for when I made my own terminal (which I've since lost) and I never had any issues with it, unless you entered in bad JavaScript code, since it was designed to directly handle code.


This is difficult to do in a general-purpose console module: To pause execution, the console has to run its own loop (not async friendly); and the modular nature of the Sphere v2 design means evaluating arbitrary JS code is of limited utility, as very little is actually reachable globally.

Quote

I thought most APIs/frameworks use events like SDL_PollEvent/SDL_KEYDOWN/SDL_KEYUP, JavaScript DOM events, etc. That was what I had in mind here.


One of Sphere's biggest strengths, and why I fell in love with it in the first place, is that it allows you to code your game as though you were writing a DOS program.  Those were the times when you could do all the polling, looping, etc. you want, in realtime, without having to worry about handling events from the operating system.  You don't even need a central event pump because as long as you flip regularly, the engine takes care of all the OS-level event handling for you and your game stays responsive.  It's of course possible to add an event system to the API, and I probably will need to do so sooner or later to pave the way for Oozaru, but I wouldn't want it to be the primary method of handling input.

You don't realize just how easy Sphere makes it for you--and this is something I didn't fully appreciate until I started working on miniSphere and had to actually implement it--until you notice that you can do this:
Code: (javascript) [Select]

while (true) {
    while (true) {
        while (true) {
            FlipScreen();
        }
    }
}


And when you close the window, Sphere will completely--and because JS is garbage collected, cleanly!--bail out of that for you.  If an event system were enforced, you'd then have two options: 1) Limit yourself to a single, central event pump that watches for Quit events and bails out when it gets one (this is similar to what the built-in Dispatch loop does), or 2) Watch for Quit events in ALL of your loops and do the unwinding yourself.  And if you forget to add a quit check in one of them, well, say hello to Task Manager. :P  Both of these approaches sharpen the learning curve significantly.
Title: Re: miniSphere 4.6.0
Post by: Eggbertx on July 14, 2017, 05:31:50 pm

Which miniSphere version?  The console has infinite priority so it should always render last.  I know I did fix a bug related to incorrect Dispatch order in 4.6, maybe that's what you're encountering:
https://github.com/fatcerberus/minisphere/commit/f88b4aa96f975d4c11030abf2e02071f25adf7d0 (https://github.com/fatcerberus/minisphere/commit/f88b4aa96f975d4c11030abf2e02071f25adf7d0)


Oops, I updated the AUR package but I forgot to install the update on Windows.
How does DOM JavaScript get away with it? And I guess I should have made this more clear, I wasn't referring to low-level event handling, I was referring more to things like input. Or at least that's why I brought this up. Detecting when a person collides with something or reaches a certain point is easy enough (that's part of the reason I created the personlib wrapper). What do you think would be a good way to easily allow for differentiating between a keydown and keyup?
Title: Re: miniSphere 4.6.0
Post by: Rhuan on July 15, 2017, 06:23:12 pm
Low level engine question:
- if you instruct miniSphere to draw a shape to the screen of which a large portion will be off the edge of the screen and just part of it on screen what happens?

- does it waste effort processing all the parts that aren't on screen? (I assume yes and hence to optimise one should try and only draw what's actually on screen in one's javascript) but don't want to waste effort arranging that if the graphics system automatically clips the excess.
Title: Re: miniSphere 4.6.0
Post by: Fat Cerberus on July 15, 2017, 09:06:38 pm

Low level engine question:
- if you instruct miniSphere to draw a shape to the screen of which a large portion will be off the edge of the screen and just part of it on screen what happens?

- does it waste effort processing all the parts that aren't on screen? (I assume yes and hence to optimise one should try and only draw what's actually on screen in one's javascript) but don't want to waste effort arranging that if the graphics system automatically clips the excess.


That's entirely up to the graphics hardware.  miniSphere just sends the vertices to the GPU (they're actually uploaded when you do new Shape() but you get the idea); because of the way projection matrices and such work there's no way for the engine to know which parts of a shape will be visible or not.  At the very least, it's been my experience that if you draw a huge number of vertices per frame, even if the vertices are already uploaded, you will experience slowdown regardless of how much of it actually gets displayed.  Vertex processing is apparently more expensive than rasterization, for whatever reason.
Title: Re: miniSphere 4.6.0
Post by: Fat Cerberus on July 16, 2017, 01:56:20 pm
I'm currently working on implementing Sphere1.x compatible spriteset behavior.  A few v1 games rely on being able to modify spritesets on-the-fly, and the internal spriteset code needs to be refactored anyway (it's some of the oldest code in the engine), so this is a good opportunity to get it out of the way.
Title: Re: miniSphere 4.6.0
Post by: Rhuan on July 16, 2017, 02:43:13 pm
Have you had a chance to look into the framerate issue where a limited framerate results in a drop from 50 FPS to 30 or the like?
Title: Re: miniSphere 4.6.0
Post by: Fat Cerberus on July 16, 2017, 02:48:42 pm

Have you had a chance to look into the framerate issue where a limited framerate results in a drop from 50 FPS to 30 or the like?


I know why it happens, I'm just not sure how to fix it yet.
See: https://github.com/fatcerberus/minisphere/issues/188
Title: Re: miniSphere 4.6.0
Post by: Rhuan on July 16, 2017, 03:57:22 pm


Have you had a chance to look into the framerate issue where a limited framerate results in a drop from 50 FPS to 30 or the like?


I know why it happens, I'm just not sure how to fix it yet.
See: https://github.com/fatcerberus/minisphere/issues/188
Average the time over 10 frames or something?
Title: Re: miniSphere 4.6.0
Post by: Fat Cerberus on July 17, 2017, 09:35:37 am

Average the time over 10 frames or something?


Yeah, I was thinking something like that would be needed.  I didn't tag the GitHub issue for v4.7.0 because I don't know how difficult it will actually be to implement.  Once I finish the spriteset refactoring I'll look into it, at least.
Title: Re: miniSphere 4.6.0
Post by: Fat Cerberus on July 17, 2017, 04:47:41 pm
I just finished implementing v1 CreateSpriteset().  I think I'll add Spriteset#save() while I'm at it.
Title: Re: miniSphere 4.6.0
Post by: Fat Cerberus on July 18, 2017, 01:25:57 pm
@Rhuan @Eggbert: I'm done with feature additions for miniSphere 4.7 for now; anything else you would like to see make it into the 4.7.0 release? (besides the frame-limiting fix; it's far from trivial to fix so I'll probably postpone it).

Here's the changelog so you can see what I've already done:
https://github.com/fatcerberus/minisphere/blob/master/CHANGELOG.md
Title: Re: miniSphere 4.6.0
Post by: Eggbertx on July 18, 2017, 04:14:42 pm
I can't really think of anything at the moment. I figured out a cheap way to prevent repeated keypress dtection for what I wanted to do (pressing tab toggles a debug popup that follows the player character), and it shouldn't be too difficult to set up a library to handle that for all of the keys.
Title: Re: miniSphere 4.6.0
Post by: Rhuan on July 18, 2017, 04:19:46 pm
I was trying to write an algorithm for the frame skipping after work yesterday but all I did was fill paper with numbers :( May just be too tired at the moment.

I've got some other ideas about optimisations but would like to test what the engine can do with the changes you've made so far before suggesting anything further, thanks for all your efforts.
Title: Re: miniSphere 4.6.0
Post by: Fat Cerberus on July 18, 2017, 04:48:38 pm

I was trying to write an algorithm for the frame skipping after work yesterday but all I did was fill paper with numbers :( May just be too tired at the moment.


One idea I had was to steal time from early frames, but that doesn't help you at all in the case where all your frames are late.  Plus it risks introducing undercrank, i.e. after a late frame the subsequent ones will be given a smaller timeslice and cause the game too fast, leading to jerky animations etc.

So yeah, it's not a trivial problem to fix. :(
Title: Re: miniSphere 4.6.0
Post by: Rhuan on July 18, 2017, 05:41:25 pm


I was trying to write an algorithm for the frame skipping after work yesterday but all I did was fill paper with numbers :( May just be too tired at the moment.


One idea I had was to steal time from early frames, but that doesn't help you at all in the case where all your frames are late.  Plus it risks introducing undercrank, i.e. after a late frame the subsequent ones will be given a smaller timeslice and cause the game too fast, leading to jerky animations etc.

So yeah, it's not a trivial problem to fix. :(
I was trying to work out a way to determine what framerate should be targeted based on the delays and how many frames had been drawn so far and hence whether to skip or not and but it felt like there was a step I was missing.
Title: Re: miniSphere 4.6.0
Post by: Fat Cerberus on July 19, 2017, 02:57:36 am
Funny thing happened: In the process of messing with the Galileo code, I discovered a bug: the Shape constructor wasn't checking for failure after uploading the vertices to the GPU.  So I fixed that... and found out that vertex upload has been broken for a long time.  Depending on the game, after a while VBO creation starts to fail.  That doesn't crash the engine because v4.6 and earlier had a fallback for older machines without VBO support.  But now the fallbacks are gone, this needs to be fixed ASAP.

I'm not sure whether I'm leaking vertex buffers, or maybe Allegro is... even so, the failure seems to occur after only a few hundred shapes, while my GTX 1070 has 8GB of RAM!  So I very much doubt I'm running out of VRAM.  What fun...
Title: Re: miniSphere 4.6.0
Post by: Fat Cerberus on July 19, 2017, 01:14:49 pm
The vertex buffer failure looks like a race condition of some kind.  If I put a breakpoint after the error, and then Set Next Statement to try creating the VBO again, it succeeds.  Then it fails again a bit down the line.  So I'm guessing it's some kind of Allegro bug, but Allegro is just doing simple bog standard GL calls:
https://github.com/liballeg/allegro5/blob/5.2.2.0/addons/primitives/prim_opengl.c#L666-L669

No idea why that should fail.  The buffers in question are tiny, between 4-100 vertices (circle fans, image blits, etc.).  Even if I ended up leaking every single buffer (which I'm not--the VBO that first fails gets ID 3), it fails after only ~150 shapes.  That's 1MB of vertices, tops.
Title: Re: miniSphere 4.6.0
Post by: Rhuan on July 19, 2017, 02:58:07 pm
Seems absolutely bizarre, could you insert an if and just try again if it fails?

Obviously a problem if it fails twice... Run a loop that ends when it succeeds? Seems rather painful, but hopefully should be rare that it takes more than 1 run of the loop?

I guess it's dangerous that if for some reason it never succeeds that loop would go forever.
Title: Re: miniSphere 4.6.0
Post by: Fat Cerberus on July 19, 2017, 03:19:20 pm
Yep, retrying the operation even once makes it work.  My theory is that Allegro is forgetting to check the result of glGetError() somewhere, causing the error to be picked up later during VBO creation.  Calling glGetError() clears the error flag, allowing the second attempt to succeed.  I've already reported the bug to the Allegro devs.
Title: Re: miniSphere 4.6.0
Post by: Fat Cerberus on July 20, 2017, 02:12:31 am
Alright, I got everything working again and did a bunch of internal refactoring to make the Galileo code easier to work with going forward and more extensible.

There's also a surprise new addition to the Galileo API (I fibbed about not adding anymore features to 4.7 ;): VertexList and IndexList objects!  These let you upload a bunch of vertices to the GPU and share them between different shapes--the VertexList contains the vertices and the IndexList specifies which vertices to actually use in the shape.  You can also modify a shape after creation now, by replacing its vertex and index buffers.  This makes Galileo much more powerful and gives it a clear advantage over Sphere v1.  There's only one fix left that I want to do, and miniSphere 4.7 should be out within the next day or two.

Here's all the new improvements to get excited about:
https://github.com/fatcerberus/minisphere/blob/master/CHANGELOG.md
Title: Re: miniSphere 4.7.0
Post by: Fat Cerberus on July 20, 2017, 09:38:55 pm
miniSphere 4.7.0 is now available for Windows and Ubuntu 16.10 and later.

This release brings performance enhancements, API refinements (per-surface clipping, e.g.), new VertexList and IndexList objects, better Sphere 1.x compatibility, and more.  Note that a shader-capable GPU is now required, so no more running the engine on old Pentium 4 budget machines. ;)
Title: Re: miniSphere 4.7.0
Post by: Rhuan on July 21, 2017, 01:42:14 am
Built for mac, not sure what I think of the VertexList facility, can't currently think where I'd use it and so at the moment it's just an extra function call when creating a shape.

It is very good to see the better draw speed though.
Title: Re: miniSphere 4.7.0
Post by: Fat Cerberus on July 21, 2017, 01:59:14 am
It's often more efficient to store a bunch of vertices in one buffer and then use an index list to take only the ones you need, rather than having a bunch of separate buffers which was always the case before.  Basically, for a bit of extra complexity in the API you get a lot more power to optimize where needed.  It's a tradeoff.

Also by swapping in a new vertex list you can redefine a shape after creation, which I believe is something you asked for before...? ;)
Title: Re: miniSphere 4.7.0
Post by: Rhuan on July 21, 2017, 02:15:15 am

It's often more efficient to store a bunch of vertices in one buffer and then use an index list to take only the ones you need, rather than having a bunch of separate buffers which was always the case before.  Basically, for a bit of extra complexity in the API you get a lot more power to optimize where needed.  It's a tradeoff.

Also by swapping in a new vertex list you can redefine a shape after creation, which I believe is something you asked for before...? ;)
I asked for it then I discovered that shaders let me do what I wanted (a) use a single texture as an atlas for a sprite and b) apply colour masks) more efficiently (or at least with a lot less lines of code).

The only use I can see for a VertexList at the moment is to optimise mode 2 of the rings test script, I'll think on it more though.
Title: Re: miniSphere 4.7.0
Post by: Eggbertx on July 21, 2017, 03:04:18 am
You mention that Pentium 4 machines are no longer compatible for obvious reasons (my home server actually has a Pentium 4 and serves Xonotic, Quake 1/3, OpenTTD, and Starbound reasonably well). But Is it possible that this would limit relatively newer PCs?



I was hoping to keep this a secret until I got some basic functionality out of them, but I was working on porting miniSphere to JavaScript via emscripten and to Android. I figure most Android phones should be ok, but do you think this would prevent emscripten/JavaScript compatibility, or make it more difficult? Duktape easily compiles but I haven't gotten Allegro to build yet.
Title: Re: miniSphere 4.7.0
Post by: Fat Cerberus on July 21, 2017, 09:09:58 am
I don't see why not, assuming you can get Allegro to compile.  The mention of P4 machines was more because such machines (at least the low- to mid-level ones) usually had only Intel onboard graphics, which meant Project64 was about the most graphics-intensive thing they could run.  No shader support or anything like that, just the basic fixed-function pipeline.  Sphere 1.x is right at home there, but supporting ancient machines in miniSphere was holding me back from doing all the stuff I wanted to do with the Sv2 graphics system.
Title: Re: miniSphere 4.7.0
Post by: Rhuan on July 21, 2017, 05:10:17 pm
I'm trying to load test my map and sprite engines, it can draw a basic map with 1200 sprites moving around on it (with pre-set movement instructions) at 55-60 FPS pretty comfortably.

BUT if I draw the map and sprites to a surface, do some transformations on the surface then convert to a texture, texture a shape with it and draw the shape (with some transformations) the framerate drops to 4-30 FPS and staggers around in that range.

Is there any other way I could be transforming the whole map every frame that may be faster? (Obviously I could try and apply the relevant transformations to everything I'm drawing thought that could be a lot trickier and slow things down for something that is likely going to be a fringe feature.)
Title: Re: miniSphere 4.7.0
Post by: Fat Cerberus on July 21, 2017, 05:42:21 pm
That's exactly why I haven't implemented SetLayerAngle/SetLayerScaleFactor yet.  Your results with rendering to a texture don't surprise me - you're talking about a HUGE texture here that there's no way the GPU can keep it all in its cache at once.

One way you could get what you want would be to build a bunch of Shapes for everything you want to render, then add them all to a Model and transform the whole thing as a unit, using a single matrix.  This, in fact, is the very reason the Model object exists.  Just keep in mind that rotation and scaling are always done about the origin--so assuming you're working in pixels, you'd need to offset your vertices by (-W/2, -H/2).  That offset could even be built into the matrix as a translation.
Title: Re: miniSphere 4.7.0
Post by: Rhuan on July 21, 2017, 06:44:19 pm
I can't do it as one model as I'm doing different uniforms for each sprite to handle their spritesheets being atlases, I guess I could do lots of different vertex sets instead or something instead of using uniforms to work the atlases BUT I think there would be several negative trade offs to that.

But I can do it a different way it will work :)
Title: Re: miniSphere 4.7.0
Post by: Eggbertx on July 22, 2017, 02:33:07 pm
I finally got around to messing with Cell, and some code that worked up until then no longer works

Code: (javascript) [Select]

const objects = require("objects");
//...
func game()
    // console.initialize();
    SetDefaultMapScript(SCRIPT_ON_ENTER_MAP,"objects.generateField(); "); // ReferenceError: identifier 'objects' undefined
}
game();

Just to make sure that objects exists at all, I put Sphere.abort(objects.abort.player.name) outside SetDefaultMapScript and it worked fine. What changed?
Title: Re: miniSphere 4.7.0
Post by: Fat Cerberus on July 22, 2017, 02:41:22 pm
If you don't set Sphere.Game.version to 1 in your Cellscript, the default is 2.  In that case your main script will be loaded as a CommonJS module (like node) rather than normal program code (like Sphere 1.x/browsers).  You have a few options: A) Set the API version to 1; B) Explicitly make the variable global global.objects = ...; or C) Take advantage of closures and pass a function to SetDefaultMapScript().
Title: Re: miniSphere 4.7.0
Post by: Fat Cerberus on July 22, 2017, 02:49:28 pm
To be clear on the behavior, when run in v2 mode, the engine boots your game by doing internally the equivalent of:
Code: (javascript) [Select]

let Game = require(mainScriptFilename).default:
if (Game) { (new Game()).start(); }
Title: Re: miniSphere 4.7.0
Post by: Fat Cerberus on July 22, 2017, 03:40:37 pm
I was experimenting with adding mp3 support using minimp3 together with the existing internal Audialis bindings and got some promising results, but found out in the process that SoundStreams are pretty much useless at present.  Because the engine is single threaded, the smallest delays will starve the stream and cause stuttering, even the intentional delays from frame rate limiting.  So I'm thinking I'll either need to move the feed into a separate thread or else increase the size of the stream buffer.
Title: Re: miniSphere 4.7.0
Post by: Eggbertx on July 22, 2017, 08:38:43 pm
Would it be unfeasible to have audio, video, and input in separate threads?
Title: Re: miniSphere 4.7.0
Post by: Fat Cerberus on July 23, 2017, 12:53:05 am
That's one option, the main reason I avoided it in the current implementation is because streaming from a separate thread requires synchronization (think mutexes) to ensure the stream buffer doesn't get clobbered.  Right now soundstream.buffer() is super-cheap because it can just stuff bytes into the buffer with reckless abandon secure in the knowledge that Audialis won't touch it until the next flip. :P  Having to take a mutex lock first will most likely hurt performance.

There is one other solution I can try first.  Sphere.sleep() still runs the event pump, and Allegro posts an event to the queue whenever a stream has a free buffer.  If I listen for that event, it might be enough to ensure the stream doesn't starve even during long delays.
Title: Re: miniSphere 4.7.0
Post by: Eggbertx on July 23, 2017, 01:49:07 am
I'm not very familiar with working with threads, but I figured syncing could be a challenge. So would Sphere.sleep() be used as or adapted to a mutex?
Title: Re: miniSphere 4.7.0
Post by: Fat Cerberus on July 23, 2017, 01:59:44 am
The sleep function wouldn't need to take the mutex, `soundstream.buffer()` would--every time you call it.  The streaming thread would need to lock the mutex every time it fed more audio data.  It's a lot of extra synchronization overhead that seems unnecessary when we already have a built-in sync point that games need to call anyway and is much easier to predict (screen.flip).  The question is whether I'm clever enough to make it work :)
Title: Re: miniSphere 4.7.0
Post by: Eggbertx on July 23, 2017, 02:17:01 am
I guess I should have made a new post rather then editing my post to avoid unintentionally giving off the wrong idea. I'm not very familiar with working with threads.
Title: Re: miniSphere 4.7.0
Post by: Fat Cerberus on July 23, 2017, 06:25:52 pm
MiniMP3 seems to be buggy or something.  Stereo files at least don't seem to be decoded properly; if I create a stereo stream with the proper frequency, the sound is choppy (like it's coming through a box fan or something) and plays at half speed.  If I create a mono stream at double the frequency, it sounds normal with just a bit of distortion.  No idea what's up with that.

In any case I most likely will ultimately use a different mp3 library.  I want to try to get Fluendo's decoder working because that's MIT-licensed so I don't have to worry about LGPL quibbles.  And all the mp3 patents are expired now so that'll be great :D
Title: Re: miniSphere 4.7.1
Post by: Fat Cerberus on July 24, 2017, 02:48:18 am
miniSphere 4.7.1 fixes a very, very old bug where getting stuck in an infinite loop caused you to be completely locked out of the debugger.  Not only does that mean you no longer have to kill the miniSphere process when that happens, it also means you can click the Pause button in Sphere Studio to find out exactly where your code is stuck!
Title: Re: miniSphere 4.7.0
Post by: Eggbertx on July 25, 2017, 02:03:01 am

If you don't set Sphere.Game.version to 1 in your Cellscript, the default is 2.  In that case your main script will be loaded as a CommonJS module (like node) rather than normal program code (like Sphere 1.x/browsers).  You have a few options: A) Set the API version to 1; B) Explicitly make the variable global global.objects = ...; or C) Take advantage of closures and pass a function to SetDefaultMapScript().


Just making sure I understand this correctly so that I avoid running into similar errors later on, "synth:onLoadMap" and any other similar case run from SetDefaultMapScript runs from a different "scope" than the main module and anything declared/defined in it that is not explicitly added to the global object? If I put
Code: [Select]

// main
global.somepkg = require('somepkg');
global.otherpkg = require('otherpkg');

is somepkg available from otherpkg without doing
Code: [Select]
global.somepkg = require('somepkg');
from there as well? I have only a rudimentary understanding of CommonJS usage.


Also, as a suggestion, can you replace glob strings in CellScript's files function with a regular expression? I don't know how common this is, but I sometimes save copies of a script as "scriptname.bak.js" if I have to make major changes to scriptname.js and don't want to have to do a lot of backtracking and rewriting if I have to exit before the major change is implemented. This would allow me to do something like
Code: [Select]
install("@/scripts",files(/scripts\/.*(^bak)\.js$/, true));
Title: Re: miniSphere 4.7.1
Post by: Fat Cerberus on July 25, 2017, 02:33:23 am
CommonJS is pretty simple to understand, just pretend the whole module is wrapped inside of:
Code: [Select]

(function() {
    // your module code here
})()


...which is exactly what the engine does under the hood when loading them.  That way any vars, functions, etc. are local to the module.  By assigning something to global you explicitly put it in global scope, the same as adding stuff to window in a browser.

SetDefaultMapScript() etc. basically work like eval(), anything in there is effectively run in the global scope.  Unlike Sphere 1.x, miniSphere lets you pass a function to them instead though, if you wanted to take advantage of normal scoping rules.

Regarding Cellscript regular expressions, I actually wanted to implement that originally, but it's pretty hard right now because files() is implemented natively and Duktape doesn't expose its regex engine to C code.  I didn't want to add a dependency on a third-party regex engine just for the sake of pattern matching files.  Maybe I'll put in a feature request on the Duktape repo...
Title: Re: miniSphere 4.7.1
Post by: Fat Cerberus on July 25, 2017, 02:36:23 am
By the way, tip: requires are cached.  So if you do require('something') in multiple places, only the first one will actually load the module and the rest will just return the cached exports.  So you get built-in lazy loading that way without needing to store it globally.
Title: Re: miniSphere 4.7.1
Post by: Eggbertx on July 25, 2017, 03:16:19 am

CommonJS is pretty simple to understand, just pretend the whole module is wrapped inside of:
Code: [Select]

(function() {
    // your module code here
})()


...which is exactly what the engine does under the hood when loading them.  That way any vars, functions, etc. are local to the module.  By assigning something to global you explicitly put it in global scope, the same as adding stuff to window in a browser.

SetDefaultMapScript() etc. basically work like eval(), anything in there is effectively run in the global scope.  Unlike Sphere 1.x, miniSphere lets you pass a function to them instead though, if you wanted to take advantage of normal scoping rules.

Alright, good to know to avoid future confusion.


Regarding Cellscript regular expressions, I actually wanted to implement that originally, but it's pretty hard right now because files() is implemented natively and Duktape doesn't expose its regex engine to C code.  I didn't want to add a dependency on a third-party regex engine just for the sake of pattern matching files.  Maybe I'll put in a feature request on the Duktape repo...

Is there no available Regex engine that can be implemented similar to Duktape (i.e. you provide the source and header files in your own code rather than linking)? It seems especially strange, considering the RegExp constructor is available.


By the way, tip: requires are cached.  So if you do require('something') in multiple places, only the first one will actually load the module and the rest will just return the cached exports.  So you get built-in lazy loading that way without needing to store it globally.

I figured that would be the case, since that would be more efficient than having to reload a library every time require is called.
Title: Re: miniSphere 4.7.1
Post by: Eggbertx on July 25, 2017, 03:18:33 am
Oh, and you never answered my question. In the meantime, until regular expressions are allowed, or if they aren't at all, is scriptname.bak.js a common practice in case a major change doesn't work out?
Title: Re: miniSphere 4.7.1
Post by: Fat Cerberus on July 25, 2017, 10:29:19 am

Oh, and you never answered my question. In the meantime, until regular expressions are allowed, or if they aren't at all, is scriptname.bak.js a common practice in case a major change doesn't work out?


I can't speak for anyone else, but I use version control for that (Git).  If I used your method though, I would probably swap it around to script.js.bak instead--then normal globbing would exclude them.  For me version control works better though because my "major changes" tend to involve multiple files (thanks to my penchant for refactoring) and at some point I would almost surely forget to back one up and screw myself over. :P
Title: Re: miniSphere 4.7.1
Post by: Eggbertx on July 25, 2017, 11:18:13 am
I have considered using version control, but .bak.* is quick and gets the job done. Also I'm not super familiar with Git commands outside of commit, push, diff, and checkout without having to look them up.
Title: Re: miniSphere 4.7.1
Post by: Fat Cerberus on July 25, 2017, 11:44:04 am
I use GitHub Desktop for day-to-day stuff and only use the command line interface for more advanced stuff (like bisect).  Nothing like being able to right-click individual files and "Discard Changes" to automatically revert to the previous commit.  It would be awesome if it let you discard changes within a file that way, that'd be perfect.

Command-line junkies can say what they want, but sometimes a GUI really is a more efficient way to work.
Title: Re: miniSphere 4.7.1
Post by: Fat Cerberus on July 25, 2017, 12:06:05 pm

Is there no available Regex engine that can be implemented similar to Duktape (i.e. you provide the source and header files in your own code rather than linking)? It seems especially strange, considering the RegExp constructor is available.


I can actually just call the regexp object methods directly (so I was wrong to imply it was impossible), but for sandboxing purposes I didn't consider that ideal.  It's a good idea if nothing else works out, though.
Title: Re: miniSphere 4.7.1
Post by: Eggbertx on July 25, 2017, 12:06:57 pm
I use Linux a lot, and according to WineHQ, it doesn't run, or runs terribly (https://appdb.winehq.org/objectManager.php?sClass=application&iId=14249).
Title: Re: miniSphere 4.7.1
Post by: Eggbertx on July 25, 2017, 12:08:36 pm
Actually, that just gave me an idea. I'll see if I can eventually have Git support in QtSphere IDE, though that's definitely not a priority right now.
Title: Re: miniSphere 4.7.1
Post by: Fat Cerberus on July 25, 2017, 12:11:36 pm

I use Linux a lot, and according to WineHQ, it doesn't run, or runs terribly (https://appdb.winehq.org/objectManager.php?sClass=application&iId=14249).


That doesn't surprise me, I'm pretty sure it's written in C#.  Sphere Studio doesn't run that well in Wine either.

Regarding Git support in the IDE, I considered that for Sphere Studio at one point too.  Maybe I'll eventually get around to implementing it. ;)
Title: Re: miniSphere 4.7.1
Post by: Rhuan on July 25, 2017, 01:52:03 pm
I'm about to disappear for a while, but advance query for when I'm back, after I get the mapengine working well I'd be keen to help with making a variant of minisphere to run with spidermonkey, I'm very curious what that would do to performance.

(Note, in my draft sprite and map engine scripts the bottleneck performance wise is now in the javascript not in the rendering - when I say bottleneck I'm talking about it slowing down when dealing with moving ~1000 sprites)
Title: Re: miniSphere 4.7.1
Post by: Fat Cerberus on July 25, 2017, 02:35:30 pm

I'm about to disappear for a while, but advance query for when I'm back, after I get the mapengine working well I'd be keen to help with making a variant of minisphere to run with spidermonkey, I'm very curious what that would do to performance.

(Note, in my draft sprite and map engine scripts the bottleneck performance wise is now in the javascript not in the rendering - when I say bottleneck I'm talking about it slowing down when dealing with moving ~1000 sprites)


Performance should be pretty close to native code as SpiderMonkey has a JIT.  Duktape just compiles to bytecode and then interprets it, so slowdown with 1,000+ entities is understandable.  It's good to hear that the JS is the bottleneck though, as it means I did something right with Galileo in 4.7 :)
Title: Re: miniSphere 4.7.1
Post by: DaVince on July 26, 2017, 03:20:21 am

I use Linux a lot, and according to WineHQ, it doesn't run, or runs terribly (https://appdb.winehq.org/objectManager.php?sClass=application&iId=14249).

It's probably a little different, but gitg (https://wiki.gnome.org/Apps/Gitg/) has served me fine for selecting what changes to commit and going through the commit history. Only thing left to do after that is git push on the terminal.
Title: Re: miniSphere 4.7.1
Post by: Eggbertx on July 26, 2017, 02:53:03 pm
Actually, I kinda forgot that Visual Studio Code (my favorite text editor/lightweight IDE) has built-in git support.

I'm about to disappear for a while, but advance query for when I'm back, after I get the mapengine working well I'd be keen to help with making a variant of minisphere to run with spidermonkey, I'm very curious what that would do to performance.

(Note, in my draft sprite and map engine scripts the bottleneck performance wise is now in the javascript not in the rendering - when I say bottleneck I'm talking about it slowing down when dealing with moving ~1000 sprites)

Performance should be pretty close to native code as SpiderMonkey has a JIT.  Duktape just compiles to bytecode and then interprets it, so slowdown with 1,000+ entities is understandable.  It's good to hear that the JS is the bottleneck though, as it means I did something right with Galileo in 4.7 :)
Does SpiderMonkey have built-in CommonJS support? If not, what about v8?
Title: Re: miniSphere 4.7.1
Post by: Rhuan on July 27, 2017, 03:12:09 am
Does SpiderMonkey have built-in CommonJS support? If not, what about v8?
A quick read around implies that neither of them have commonjs built in. Also they're both c++ rather than c and so some wrapper code will be required to make the minisphere code base use either of them.
Title: Re: miniSphere 4.7.1
Post by: Eggbertx on July 27, 2017, 12:05:02 pm
Oh boy, that sounds like fun.
Title: Re: miniSphere 4.7.1
Post by: Fat Cerberus on July 27, 2017, 12:15:33 pm
The lack of CommonJS support isn't a problem, CJS isn't hard to implement, you could even do it directly in JS if you wanted to.  You just wrap the text of the module in
Code: (javascript) [Select]
(function(exports, require, module, __filename, __dirname) {
    // your module here
})
and then call the function with the relevant parameters; the trickiest part is implementing require() in C because it's context-sensitive (path is relative to module calling it).  Other than that it's fairly painless.

The hard part, and the reason I keep putting it off, is like Rhuan says, both major JS engines have a C++ API so making them interoperate with miniSphere which is written in C isn't exactly trivial.  It can be done, but it's a big investment of time.
Title: Re: miniSphere 4.7.1
Post by: Eggbertx on July 27, 2017, 01:56:23 pm
Is it an issue of writing a lot of boilerplate code, or would it be more complicated than that?
Title: Re: miniSphere 4.7.1
Post by: Fat Cerberus on July 27, 2017, 02:20:00 pm
Not really complicated, just tedious.  I'd basically have to wrap almost all of the SpiderMonkey API in C functions that could be called from miniSphere's C code.  That's part of why I'm trying to isolate the script layer as much as possible from the actual engine code (which has mostly been accomplished, save for the map engine), so that only the files directly related to running JS code need to be updated, stuff like image handling, etc. don't have to be touched.
Title: Re: miniSphere 4.7.1
Post by: Rhuan on July 27, 2017, 07:03:11 pm
The hard part, and the reason I keep putting it off, is like Rhuan says, both major JS engines have a C++ API so making them interoperate with miniSphere which is written in C isn't exactly trivial.  It can be done, but it's a big investment of time.
Happy to go halves with you on the effort for this one - assuming it can wait until around september I won't have much time until then.
Title: Re: miniSphere 4.7.2
Post by: Fat Cerberus on July 27, 2017, 11:59:10 pm
miniSphere 4.7.2 is up, fixing a fair number of bugs and polishing up a few dark corners of the API.  Upgrading is highly recommended.
Title: Re: miniSphere 4.7.2
Post by: Eggbertx on July 28, 2017, 02:22:09 am
I'm not familiar with SpiderMonkey or with wrapping C around C++, but I'll look into that now.
Title: Re: miniSphere 4.7.2
Post by: Eggbertx on July 28, 2017, 11:46:58 pm
It looks like sphere2-core-api.txt isn't entirely up to date. It still says
FileStream#size [read-only]
Title: Re: miniSphere 4.7.2
Post by: Fat Cerberus on July 28, 2017, 11:48:26 pm
It looks like sphere2-core-api.txt isn't entirely up to date. It still says
FileStream#size [read-only]

Thanks, I noticed that this morning and was going to fix it, and then forgot about it.  Good thing you reminded me. :))
Title: Re: miniSphere 4.7.2
Post by: Fat Cerberus on August 04, 2017, 01:56:29 pm
I'm currently working on cleaning up the map engine code and generalizing it so I can move the API stubs into vanilla.c.  Ultimately what I'm trying to accomplish is to isolate all the Duktape-specific code to one place so it will be easier to swap in SpiderMonkey down the road.  That  still won't be a simple task, but detangling the spaghetti code certainly doesn't hurt. :P
Title: Re: miniSphere 4.7.2
Post by: Fat Cerberus on August 07, 2017, 11:35:12 am
Map engine cleanup is done, and here's the entire Sphere v1 API (minus a function here and there) in all its epic glory:
https://github.com/fatcerberus/minisphere/blob/master/src/minisphere/vanilla.c#L27-L455

274 global functions.  If you count object methods, there are over 400 functions in total.  I'm in utter shock that it's this huge, that's insane.

Pegasus (Sphere v2) is significantly more streamlined:
https://github.com/fatcerberus/minisphere/blob/master/src/minisphere/pegasus.c#L186-L381

No global functions at all (unless you count require()) :D
Title: Re: miniSphere 4.7.2
Post by: Eggbertx on August 08, 2017, 07:55:46 pm
I don't mean to toot my own horn, but do you think it might be beneficial to include Personlib in the system modules included with miniSphere? I've converted it to a mjs module and it's almost complete. I just need to make a few tweaks and it'll be "done".
Title: Re: miniSphere 4.7.2
Post by: Fat Cerberus on August 08, 2017, 08:09:05 pm
Hm... I wouldn't be against it, although it might become obsolete once the new map engine makes it in.  Still would be nice to have a more object-oriented interface for map persons though, so sure, that'd be awesome. :)

I'm planning to release miniSphere 4.8.0 on 8/12 so if you have it ready before then I can include it, as long as you're okay with licensing it under miniSphere's BSD-3 license.
Title: Re: miniSphere 4.7.2
Post by: Eggbertx on August 08, 2017, 08:27:21 pm
I just have to add a couple helper methods and check some things. Since it's really just a wrapper around the Person related 1.x functions, any bugs would be the fault of the engine, rather than my code.

As far as licensing, almost everything I do is licensed under the BSD 2 clause license for simplicity. And because I have some gripes with the GPL. So I'd be fine with the 3-clause license. Are there any major differences?
Title: Re: miniSphere 4.7.2
Post by: Fat Cerberus on August 08, 2017, 08:41:57 pm
I dislike the GPL too.  LGPL is better, but still a bit too "viral" for my tastes.  They work well for applications, but it gets ugly with libraries.

I think the only difference between BSD 2- and 3-clause is the third clause, you can't use the name of the project or contributors to advertise derivatives.  Or something like that, I'm not actually sure what it means in practice, it seems like something akin to trademark protection?  I probably should just relicense to BSD-2 or MIT, it'd be simpler.
Title: Re: miniSphere 4.7.2
Post by: Eggbertx on August 08, 2017, 08:48:24 pm
One of my biggest issues with the GPL is the linking requirement. One of my first experiences with programming was writing JavaScript plugins for an IRC bot that was licensed under the GPL. I couldn't understand why user-made plugins had to be kept outside the bot's base folder in a folder called "xplugins" at the time, and it left a bad taste in my mouth. The GPL had its purpose at one point, but I can't see myself ever using it, even for programs that won't have any user-made plugins or libraries.

And considering this is just a helper module, that third clause doesn't really bother me.
Title: Re: miniSphere 4.7.2
Post by: Eggbertx on August 08, 2017, 10:07:42 pm
Alright, Personlib is pretty stable now and ready for inclusion.
http://forums.spheredev.org/index.php?action=dlattach;topic=1494.0;attach=823
Title: Re: miniSphere 4.7.2
Post by: Fat Cerberus on August 09, 2017, 02:31:48 pm
I had to make a few modifications to get it to run in miniSphere.  I'll experiment and kick the tires a bit later, definitely looks useful though!  I wonder now just how far it would be possible to make an object-oriented wrapper around the v1 map engine, without rewriting it from the ground up...

Thanks for this, it should come in handy. :)

Note for future:
System modules can't currently be .mjs because they are out of the purview of a Cellscript and Duktape doesn't support ES6.  I had to convert it back to CommonJS and edit out the ES6 syntax (I think you have a for-of loop in there, no other issues I saw) to get it to work.
Title: Re: miniSphere 4.7.2
Post by: Eggbertx on August 09, 2017, 03:11:24 pm
I was actually considering creating a new object oriented wrapper for the map engine, but I figured it wouldn't be much of a benefit because of the way the engine works.
The main reason I made the Person module was because having to constantly call DoSomethingWithPerson(...) with the person name as a parameter just seemed like boilerplate code after a certain point. I didn't originally create Personlib for MonoBomber though. I was working on a SHMUP, and I got the idea while creating the randomly generated enemies and dynamic objects (items, special traps, etc). That way I could just create a given number of enemies and dynamic objects, add them to their respective arrays, and just call methods and such for each one. It also allowed me to use the thread module, which I felt was much nicer and more organized than going through each person in a single update/render function.

I wasn't really sure if it would work as a system module, but working on it as a .mjs module made some things simpler for me. I'll keep that in mind though when posting them here.
Overall, I'm glad you like it, and I'm sure people will like using it for the same reasons that I created it.
Title: Re: miniSphere 4.7.2
Post by: Fat Cerberus on August 10, 2017, 12:47:38 am
There's a really annoying bug in the SoundStream object that I'm trying to fix, but having no luck.  I'm not sure if it's a bug in Allegro or if maybe I'm doing something wrong, or what, but audio for whatever reason isn't playing at the correct rate.  During my experiments to get mp3 support working, I wrote the following code:

Code: (JavaScript) [Select]
window = global;
RequireScript('@/lib/aurora.js');
RequireScript('@/lib/mp3.js');

let fs = new FileStream('music/chartreuse.mp3', FileOp.Read);
let mp3Data = fs.read(fs.fileSize);
let asset = AV.Asset.fromBuffer(mp3Data);
let stream = null;
asset.get('format', format => {
        // note: Aurora.js always decodes to float32
stream = new SoundStream(format.sampleRate, 32,  // 32bit = float32
                         format.channelsPerFrame);
});
while (stream == null && Sphere.run());
stream.play(Mixer.Default);
asset.on('data', buffer => {
stream.buffer(buffer);
Sphere.sleep(0.005);  // run event pump
});
asset.start();

It produces weird distortion like the music is being played through a box fan, and it plays at only half speed too.  However, if I change the stream creation to:
Code: (JavaScript) [Select]
stream = new SoundStream(format.sampleRate * 2, 32,  // 32bit = float32
                         format.channelsPerFrame / 2);

Then it sounds tinny, but normal speed and with no other distortion.  Very strange.
Title: Re: miniSphere 4.7.2
Post by: Eggbertx on August 10, 2017, 05:26:10 am
This isn't related to the SoundStream issue, but would it be unreasonably difficult to replace Allegro with SDL? It would make it much easier to port, and SDL seems to be more supported. I was playing with this myself a while ago, but I never got very far.
Title: Re: miniSphere 4.7.2
Post by: Fat Cerberus on August 10, 2017, 09:50:53 am
That's a lot of code to rewrite.  I have been working to get the code better organized (the 1.0 code was a mess because I coded the whole engine in like 2 months), so maybe it wouldn't be as difficult?  Still a ton of work, though.

Allegro has a lot of conveniences built-in (like shader support, render-to-texture, etc.) that I wouldn't want to lose, though.  I would have to research if SDL has the same.  I find OpenGL to be tedious to work with by hand.
Title: Re: miniSphere 4.7.2
Post by: Fat Cerberus on August 11, 2017, 08:07:26 pm
miniSphere 8/12 Big Fat Eaty Pig Edition, a.k.a. the annual 8/12 release, is out tomorrow, and on the first anniversary of miniSphere 4.0 to boot! (4.0 being the first Sphere v2 release).

This release will bring a new Image class which attempts to bring back some of the convenience of working with bitmaps that we had in sphere 1.x.  It'll be very basic in miniSphere 4.8, only supporting standard blitting, but I hope to flesh it out as time goes on.

So yeah, get ready for another awesome release!
Title: Re: miniSphere 4.8.0
Post by: Fat Cerberus on August 11, 2017, 11:59:18 pm
miniSphere 4.8.0 has been posted, get it while it's hot 8)
Title: Re: miniSphere 4.8.0
Post by: Eggbertx on August 13, 2017, 02:22:46 am
After updating, Sphere Studio wouldn't start and didn't display any errors or indication that it ran all, so I ran it via Cygwin. I then got this:
Code: [Select]
Unhandled Exception: System.Configuration.ConfigurationErrorsException: Configuration system failed to initialize ---> System.Configuration.ConfigurationErrorsException: Unrecognized configuration section userSettings/SphereStudio.Properties.Settings. (C:\Users\josh\AppData\Local\Sphere_Engine_Group\Sphere_Studio.exe_Url_h2javz01e0mxoxhijn1m1blnjaoj5uuy\0.0.0.0\user.config line 4)
   at System.Configuration.ConfigurationSchemaErrors.ThrowIfErrors(Boolean ignoreLocal)
   at System.Configuration.BaseConfigurationRecord.ThrowIfParseErrors(ConfigurationSchemaErrors schemaErrors)
   at System.Configuration.ClientConfigurationSystem.OnConfigRemoved(Object sender, InternalConfigEventArgs e)
   --- End of inner exception stack trace ---
   at System.Configuration.ClientConfigurationSystem.OnConfigRemoved(Object sender, InternalConfigEventArgs e)
   at System.Configuration.Internal.InternalConfigRoot.OnConfigRemoved(InternalConfigEventArgs e)
   at System.Configuration.Internal.InternalConfigRoot.RemoveConfigImpl(String configPath, BaseConfigurationRecord configRecord)
   at System.Configuration.BaseConfigurationRecord.GetSectionRecursive(String configKey, Boolean getLkg, Boolean checkPermission, Boolean getRuntimeObject, Boolean requestIsHere, Object& result, Object& resultRuntimeObject)
   at System.Configuration.BaseConfigurationRecord.GetSection(String configKey)
   at System.Configuration.ConfigurationManager.GetSection(String sectionName)
   at WeifenLuo.WinFormsUI.Docking.PatchController.get_EnableFontInheritanceFix()
   at WeifenLuo.WinFormsUI.Docking.DockContent..ctor()
   at WeifenLuo.WinFormsUI.Docking.DockPanel..ctor()
   at SphereStudio.Ide.IdeWindow.InitializeComponent()
   at SphereStudio.Ide.IdeWindow..ctor()
   at SphereStudio.Ide.Program.Main(String[] args)


I deleted C:\Users\josh\AppData\Local\Sphere_Engine_Group\ (and C:\Users\josh\AppData\Local\Sphereical, which I suspect may have been from an old version) and ran it again, and got this exception/error when I started it
Code: [Select]
See the end of this message for details on invoking 
just-in-time (JIT) debugging instead of this dialog box.

************** Exception Text **************
System.NullReferenceException: Object reference not set to an instance of an object.
   at SphereStudio.Ide.Forms.ConfigManager.UpdatePluginsList()
   at SphereStudio.Ide.Forms.ConfigManager..ctor()
   at SphereStudio.Ide.IdeWindow.menuConfigManager_Click(Object sender, EventArgs e)
   at SphereStudio.Ide.IdeWindow.IDEForm_Shown(Object sender, EventArgs e)
   at System.Windows.Forms.Form.OnShown(EventArgs e)
   at System.Windows.Forms.Control.InvokeMarshaledCallbackHelper(Object obj)
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Windows.Forms.Control.InvokeMarshaledCallback(ThreadMethodEntry tme)
   at System.Windows.Forms.Control.InvokeMarshaledCallbacks()


************** Loaded Assemblies **************
mscorlib
    Assembly Version: 4.0.0.0
    Win32 Version: 4.7.2101.1 built by: NET47REL1LAST
    CodeBase: file:///C:/Windows/Microsoft.NET/Framework64/v4.0.30319/mscorlib.dll
----------------------------------------
Sphere Studio
    Assembly Version: 0.0.0.0
    Win32 Version: 0.0.0.0
    CodeBase: file:///C:/Program%20Files/miniSphere/ide/Sphere%20Studio.exe
----------------------------------------
System.Windows.Forms
    Assembly Version: 4.0.0.0
    Win32 Version: 4.6.1586.0 built by: NETFXREL2
    CodeBase: file:///C:/WINDOWS/Microsoft.Net/assembly/GAC_MSIL/System.Windows.Forms/v4.0_4.0.0.0__b77a5c561934e089/System.Windows.Forms.dll
----------------------------------------
System
    Assembly Version: 4.0.0.0
    Win32 Version: 4.6.1647.0 built by: NETFXREL3STAGE
    CodeBase: file:///C:/WINDOWS/Microsoft.Net/assembly/GAC_MSIL/System/v4.0_4.0.0.0__b77a5c561934e089/System.dll
----------------------------------------
System.Drawing
    Assembly Version: 4.0.0.0
    Win32 Version: 4.6.1586.0 built by: NETFXREL2
    CodeBase: file:///C:/WINDOWS/Microsoft.Net/assembly/GAC_MSIL/System.Drawing/v4.0_4.0.0.0__b03f5f7f11d50a3a/System.Drawing.dll
----------------------------------------
SphereStudio.Base
    Assembly Version: 0.0.0.0
    Win32 Version: 0.0.0.0
    CodeBase: file:///C:/Program%20Files/miniSphere/ide/SphereStudio.Base.DLL
----------------------------------------
System.Core
    Assembly Version: 4.0.0.0
    Win32 Version: 4.6.1647.0 built by: NETFXREL3STAGE
    CodeBase: file:///C:/WINDOWS/Microsoft.Net/assembly/GAC_MSIL/System.Core/v4.0_4.0.0.0__b77a5c561934e089/System.Core.dll
----------------------------------------
WeifenLuo.WinFormsUI.Docking
    Assembly Version: 2.16.0.0
    Win32 Version: 2.16.0.0
    CodeBase: file:///C:/Program%20Files/miniSphere/ide/WeifenLuo.WinFormsUI.Docking.DLL
----------------------------------------
System.Configuration
    Assembly Version: 4.0.0.0
    Win32 Version: 4.6.1586.0 built by: NETFXREL2
    CodeBase: file:///C:/WINDOWS/Microsoft.Net/assembly/GAC_MSIL/System.Configuration/v4.0_4.0.0.0__b03f5f7f11d50a3a/System.Configuration.dll
----------------------------------------
System.Xml
    Assembly Version: 4.0.0.0
    Win32 Version: 4.6.1586.0 built by: NETFXREL2
    CodeBase: file:///C:/WINDOWS/Microsoft.Net/assembly/GAC_MSIL/System.Xml/v4.0_4.0.0.0__b77a5c561934e089/System.Xml.dll
----------------------------------------
AudioPlayerPlugin
    Assembly Version: 1.0.0.0
    Win32 Version: 1.0.0.0
    CodeBase: file:///C:/Program%20Files/miniSphere/ide/Plugins/AudioPlayerPlugin.dll
----------------------------------------
FontEditPlugin
    Assembly Version: 1.0.0.0
    Win32 Version: 1.0.0.0
    CodeBase: file:///C:/Program%20Files/miniSphere/ide/Plugins/FontEditPlugin.dll
----------------------------------------
ImageEditPlugin
    Assembly Version: 1.0.0.0
    Win32 Version: 1.0.0.0
    CodeBase: file:///C:/Program%20Files/miniSphere/ide/Plugins/ImageEditPlugin.dll
----------------------------------------
LegacySpherePlugin
    Assembly Version: 1.0.0.0
    Win32 Version: 1.0.0.0
    CodeBase: file:///C:/Program%20Files/miniSphere/ide/Plugins/LegacySpherePlugin.dll
----------------------------------------
MapEditPlugin
    Assembly Version: 1.0.0.0
    Win32 Version: 1.0.0.0
    CodeBase: file:///C:/Program%20Files/miniSphere/ide/Plugins/MapEditPlugin.dll
----------------------------------------
miniSphereGdkPlugin
    Assembly Version: 4.8.0.2230
    Win32 Version: 4.8.0.2230
    CodeBase: file:///C:/Program%20Files/miniSphere/ide/Plugins/miniSphereGdkPlugin.dll
----------------------------------------
ScriptEditPlugin
    Assembly Version: 1.0.0.0
    Win32 Version: 1.0.0.0
    CodeBase: file:///C:/Program%20Files/miniSphere/ide/Plugins/ScriptEditPlugin.dll
----------------------------------------
SoundTestPlugin
    Assembly Version: 1.0.0.0
    Win32 Version: 1.0.0.0
    CodeBase: file:///C:/Program%20Files/miniSphere/ide/Plugins/SoundTestPlugin.dll
----------------------------------------
SphereStudio.Base
    Assembly Version: 0.0.0.0
    Win32 Version: 0.0.0.0
    CodeBase: file:///C:/Program%20Files/miniSphere/ide/Plugins/SphereStudio.Base.dll
----------------------------------------
SpritesetEditPlugin
    Assembly Version: 1.0.0.0
    Win32 Version: 1.0.0.0
    CodeBase: file:///C:/Program%20Files/miniSphere/ide/Plugins/SpritesetEditPlugin.dll
----------------------------------------
TaskListPlugin
    Assembly Version: 1.0.0.0
    Win32 Version: 1.0.0.0
    CodeBase: file:///C:/Program%20Files/miniSphere/ide/Plugins/TaskListPlugin.dll
----------------------------------------
WindowstyleEditPlugin
    Assembly Version: 1.0.0.0
    Win32 Version: 1.0.0.0
    CodeBase: file:///C:/Program%20Files/miniSphere/ide/Plugins/WindowstyleEditPlugin.dll
----------------------------------------
ObjectListView
    Assembly Version: 2.9.1.1072
    Win32 Version: 2.9.1.0
    CodeBase: file:///C:/Program%20Files/miniSphere/ide/ObjectListView.DLL
----------------------------------------

************** JIT Debugging **************
To enable just-in-time (JIT) debugging, the .config file for this
application or computer (machine.config) must have the
jitDebugging value set in the system.windows.forms section.
The application must also be compiled with debugging
enabled.

For example:

<configuration>
    <system.windows.forms jitDebugging="true" />
</configuration>

When JIT debugging is enabled, any unhandled exception
will be sent to the JIT debugger registered on the computer
rather than be handled by this dialog box.
so I clicked continue and went on my merry way.

Unfortunately, when I click to open any of the reference files from the Help menu, I get "Sphere Studio doesn't know how to open that type of file and no default file opener is available. Tip: Open Configuration Manager and check your plugins."
But if I try to open the configuration manager, I ge this:
Code: [Select]
See the end of this message for details on invoking 
just-in-time (JIT) debugging instead of this dialog box.

************** Exception Text **************
System.NullReferenceException: Object reference not set to an instance of an object.
   at SphereStudio.Ide.Forms.ConfigManager.UpdatePluginsList()
   at SphereStudio.Ide.Forms.ConfigManager..ctor()
   at SphereStudio.Ide.IdeWindow.menuConfigManager_Click(Object sender, EventArgs e)
   at System.Windows.Forms.ToolStripItem.RaiseEvent(Object key, EventArgs e)
   at System.Windows.Forms.ToolStripMenuItem.OnClick(EventArgs e)
   at System.Windows.Forms.ToolStripItem.HandleClick(EventArgs e)
   at System.Windows.Forms.ToolStripItem.HandleMouseUp(MouseEventArgs e)
   at System.Windows.Forms.ToolStrip.OnMouseUp(MouseEventArgs mea)
   at System.Windows.Forms.ToolStripDropDown.OnMouseUp(MouseEventArgs mea)
   at System.Windows.Forms.Control.WmMouseUp(Message& m, MouseButtons button, Int32 clicks)
   at System.Windows.Forms.Control.WndProc(Message& m)
   at System.Windows.Forms.ToolStrip.WndProc(Message& m)
   at System.Windows.Forms.ToolStripDropDown.WndProc(Message& m)
   at System.Windows.Forms.NativeWindow.Callback(IntPtr hWnd, Int32 msg, IntPtr wparam, IntPtr lparam)


************** Loaded Assemblies **************
mscorlib
    Assembly Version: 4.0.0.0
    Win32 Version: 4.7.2101.1 built by: NET47REL1LAST
    CodeBase: file:///C:/Windows/Microsoft.NET/Framework64/v4.0.30319/mscorlib.dll
----------------------------------------
Sphere Studio
    Assembly Version: 0.0.0.0
    Win32 Version: 0.0.0.0
    CodeBase: file:///C:/Program%20Files/miniSphere/ide/Sphere%20Studio.exe
----------------------------------------
System.Windows.Forms
    Assembly Version: 4.0.0.0
    Win32 Version: 4.6.1586.0 built by: NETFXREL2
    CodeBase: file:///C:/WINDOWS/Microsoft.Net/assembly/GAC_MSIL/System.Windows.Forms/v4.0_4.0.0.0__b77a5c561934e089/System.Windows.Forms.dll
----------------------------------------
System
    Assembly Version: 4.0.0.0
    Win32 Version: 4.6.1647.0 built by: NETFXREL3STAGE
    CodeBase: file:///C:/WINDOWS/Microsoft.Net/assembly/GAC_MSIL/System/v4.0_4.0.0.0__b77a5c561934e089/System.dll
----------------------------------------
System.Drawing
    Assembly Version: 4.0.0.0
    Win32 Version: 4.6.1586.0 built by: NETFXREL2
    CodeBase: file:///C:/WINDOWS/Microsoft.Net/assembly/GAC_MSIL/System.Drawing/v4.0_4.0.0.0__b03f5f7f11d50a3a/System.Drawing.dll
----------------------------------------
SphereStudio.Base
    Assembly Version: 0.0.0.0
    Win32 Version: 0.0.0.0
    CodeBase: file:///C:/Program%20Files/miniSphere/ide/SphereStudio.Base.DLL
----------------------------------------
System.Core
    Assembly Version: 4.0.0.0
    Win32 Version: 4.6.1647.0 built by: NETFXREL3STAGE
    CodeBase: file:///C:/WINDOWS/Microsoft.Net/assembly/GAC_MSIL/System.Core/v4.0_4.0.0.0__b77a5c561934e089/System.Core.dll
----------------------------------------
WeifenLuo.WinFormsUI.Docking
    Assembly Version: 2.16.0.0
    Win32 Version: 2.16.0.0
    CodeBase: file:///C:/Program%20Files/miniSphere/ide/WeifenLuo.WinFormsUI.Docking.DLL
----------------------------------------
System.Configuration
    Assembly Version: 4.0.0.0
    Win32 Version: 4.6.1586.0 built by: NETFXREL2
    CodeBase: file:///C:/WINDOWS/Microsoft.Net/assembly/GAC_MSIL/System.Configuration/v4.0_4.0.0.0__b03f5f7f11d50a3a/System.Configuration.dll
----------------------------------------
System.Xml
    Assembly Version: 4.0.0.0
    Win32 Version: 4.6.1586.0 built by: NETFXREL2
    CodeBase: file:///C:/WINDOWS/Microsoft.Net/assembly/GAC_MSIL/System.Xml/v4.0_4.0.0.0__b77a5c561934e089/System.Xml.dll
----------------------------------------
AudioPlayerPlugin
    Assembly Version: 1.0.0.0
    Win32 Version: 1.0.0.0
    CodeBase: file:///C:/Program%20Files/miniSphere/ide/Plugins/AudioPlayerPlugin.dll
----------------------------------------
FontEditPlugin
    Assembly Version: 1.0.0.0
    Win32 Version: 1.0.0.0
    CodeBase: file:///C:/Program%20Files/miniSphere/ide/Plugins/FontEditPlugin.dll
----------------------------------------
ImageEditPlugin
    Assembly Version: 1.0.0.0
    Win32 Version: 1.0.0.0
    CodeBase: file:///C:/Program%20Files/miniSphere/ide/Plugins/ImageEditPlugin.dll
----------------------------------------
LegacySpherePlugin
    Assembly Version: 1.0.0.0
    Win32 Version: 1.0.0.0
    CodeBase: file:///C:/Program%20Files/miniSphere/ide/Plugins/LegacySpherePlugin.dll
----------------------------------------
MapEditPlugin
    Assembly Version: 1.0.0.0
    Win32 Version: 1.0.0.0
    CodeBase: file:///C:/Program%20Files/miniSphere/ide/Plugins/MapEditPlugin.dll
----------------------------------------
miniSphereGdkPlugin
    Assembly Version: 4.8.0.2230
    Win32 Version: 4.8.0.2230
    CodeBase: file:///C:/Program%20Files/miniSphere/ide/Plugins/miniSphereGdkPlugin.dll
----------------------------------------
ScriptEditPlugin
    Assembly Version: 1.0.0.0
    Win32 Version: 1.0.0.0
    CodeBase: file:///C:/Program%20Files/miniSphere/ide/Plugins/ScriptEditPlugin.dll
----------------------------------------
SoundTestPlugin
    Assembly Version: 1.0.0.0
    Win32 Version: 1.0.0.0
    CodeBase: file:///C:/Program%20Files/miniSphere/ide/Plugins/SoundTestPlugin.dll
----------------------------------------
SphereStudio.Base
    Assembly Version: 0.0.0.0
    Win32 Version: 0.0.0.0
    CodeBase: file:///C:/Program%20Files/miniSphere/ide/Plugins/SphereStudio.Base.dll
----------------------------------------
SpritesetEditPlugin
    Assembly Version: 1.0.0.0
    Win32 Version: 1.0.0.0
    CodeBase: file:///C:/Program%20Files/miniSphere/ide/Plugins/SpritesetEditPlugin.dll
----------------------------------------
TaskListPlugin
    Assembly Version: 1.0.0.0
    Win32 Version: 1.0.0.0
    CodeBase: file:///C:/Program%20Files/miniSphere/ide/Plugins/TaskListPlugin.dll
----------------------------------------
WindowstyleEditPlugin
    Assembly Version: 1.0.0.0
    Win32 Version: 1.0.0.0
    CodeBase: file:///C:/Program%20Files/miniSphere/ide/Plugins/WindowstyleEditPlugin.dll
----------------------------------------
ObjectListView
    Assembly Version: 2.9.1.1072
    Win32 Version: 2.9.1.0
    CodeBase: file:///C:/Program%20Files/miniSphere/ide/ObjectListView.DLL
----------------------------------------
ScintillaNET
    Assembly Version: 3.6.3.0
    Win32 Version: 3.6.3.0
    CodeBase: file:///C:/Program%20Files/miniSphere/ide/ScintillaNET.DLL
----------------------------------------
SphereStudio.Vanilla
    Assembly Version: 0.0.0.0
    Win32 Version: 0.0.0.0
    CodeBase: file:///C:/Program%20Files/miniSphere/ide/SphereStudio.Vanilla.DLL
----------------------------------------
SourcemapToolkit.SourcemapParser
    Assembly Version: 1.0.0.0
    Win32 Version: 1.0.0.0
    CodeBase: file:///C:/Program%20Files/miniSphere/ide/SourcemapToolkit.SourcemapParser.DLL
----------------------------------------
Microsoft.CSharp
    Assembly Version: 4.0.0.0
    Win32 Version: 4.6.1586.0
    CodeBase: file:///C:/WINDOWS/Microsoft.Net/assembly/GAC_MSIL/Microsoft.CSharp/v4.0_4.0.0.0__b03f5f7f11d50a3a/Microsoft.CSharp.dll
----------------------------------------
System.Dynamic
    Assembly Version: 4.0.0.0
    Win32 Version: 4.6.1586.0
    CodeBase: file:///C:/WINDOWS/Microsoft.Net/assembly/GAC_MSIL/System.Dynamic/v4.0_4.0.0.0__b03f5f7f11d50a3a/System.Dynamic.dll
----------------------------------------
Anonymously Hosted DynamicMethods Assembly
    Assembly Version: 0.0.0.0
    Win32 Version: 4.7.2101.1 built by: NET47REL1LAST
    CodeBase: file:///C:/WINDOWS/Microsoft.Net/assembly/GAC_64/mscorlib/v4.0_4.0.0.0__b77a5c561934e089/mscorlib.dll
----------------------------------------
Newtonsoft.Json
    Assembly Version: 10.0.0.0
    Win32 Version: 10.0.3.21018
    CodeBase: file:///C:/Program%20Files/miniSphere/ide/Newtonsoft.Json.DLL
----------------------------------------
System.Numerics
    Assembly Version: 4.0.0.0
    Win32 Version: 4.6.1586.0 built by: NETFXREL2
    CodeBase: file:///C:/WINDOWS/Microsoft.Net/assembly/GAC_MSIL/System.Numerics/v4.0_4.0.0.0__b77a5c561934e089/System.Numerics.dll
----------------------------------------
System.Runtime.Serialization
    Assembly Version: 4.0.0.0
    Win32 Version: 4.6.1637.0 built by: NETFXREL3STAGE
    CodeBase: file:///C:/WINDOWS/Microsoft.Net/assembly/GAC_MSIL/System.Runtime.Serialization/v4.0_4.0.0.0__b77a5c561934e089/System.Runtime.Serialization.dll
----------------------------------------
System.Data
    Assembly Version: 4.0.0.0
    Win32 Version: 4.6.1636.0 built by: NETFXREL3STAGE
    CodeBase: file:///C:/WINDOWS/Microsoft.Net/assembly/GAC_64/System.Data/v4.0_4.0.0.0__b77a5c561934e089/System.Data.dll
----------------------------------------

************** JIT Debugging **************
To enable just-in-time (JIT) debugging, the .config file for this
application or computer (machine.config) must have the
jitDebugging value set in the system.windows.forms section.
The application must also be compiled with debugging
enabled.

For example:

<configuration>
    <system.windows.forms jitDebugging="true" />
</configuration>

When JIT debugging is enabled, any unhandled exception
will be sent to the JIT debugger registered on the computer
rather than be handled by this dialog box.
Title: Re: miniSphere 4.8.0
Post by: Fat Cerberus on August 13, 2017, 02:25:54 am
Navigate to %LOCALAPPDATA% and delete Sphere_Engine_Group and Spherical folders, if they exist.  The .NET settings manager apparently doesn't like applications having their metadata changed.

Sorry about that, I ran into the same issues during development, fixed it, and then forgot all about it. :-[

edit: Nevermind, you did that already.  Hm... a fresh install of 4.8.0 seems to work here on a quick test, but I'll do some more digging.

edit 2: It *MIGHT* be caused by an old copy of the miniSphere plugin that got left over from a previous installation somewhere.  I thought I made the plugin loader resilient enough to deal with bad/invalid plugins without crashing, but maybe not...
Title: Re: miniSphere 4.8.0
Post by: DaVince on August 13, 2017, 02:48:15 am
Nice work on the new version! Does this also include the new directory/file API?

Also, do you have Twitter? I could add you to the @spheredev Twitter account so you can send out updates on new releases if you like. (Same for Facebook, really)
Title: Re: miniSphere 4.8.0
Post by: Fat Cerberus on August 13, 2017, 02:53:14 am
Does this also include the new directory/file API?

Yep, it was a last-minute addition to 4.8 :D

Here's the DirectoryStream API:
https://github.com/fatcerberus/minisphere/blob/v4.8.0/docs/sphere2-core-api.txt#L365-L429

And FS.fullPath() which is really useful if you want to have default directories like in Sphere v1:
https://github.com/fatcerberus/minisphere/blob/v4.8.0/docs/sphere2-core-api.txt#L524-L534
Title: Re: miniSphere 4.8.0
Post by: Fat Cerberus on August 13, 2017, 02:55:05 am
Nice work on the new version! Does this also include the new directory/file API?

Also, do you have Twitter? I could add you to the @spheredev Twitter account so you can send out updates on new releases if you like. (Same for Facebook, really)

I have a Twitter, I never use it though.  @fattycerberus

I have a Facebook too, but I gave up on it because Facebook keeps suspending my account due to "suspicious activity" and makes me upload a selfie.  Repeatedly.  10 seconds after I regain access (mind you it usually takes 3 days to regain access).
Title: Re: miniSphere 4.8.0
Post by: Eggbertx on August 13, 2017, 03:11:19 am
So should I uninstall it again and go through to make sure that absolutely everything is gone?
Title: Re: miniSphere 4.8.0
Post by: Fat Cerberus on August 13, 2017, 03:25:51 am
So should I uninstall it again and go through to make sure that absolutely everything is gone?

Yeah, uninstall-reinstall is my general recommendation for big releases like this anyway.  Inno Setup is really bad at removing old files so they just pile up until you do a complete uninstall (at which point, then it decides to remove all the old stuff--go figure).

There also might be an old miniSphere plugin in C:\ProgramData somewhere.  If so, delete it.
Title: Re: miniSphere 4.8.0
Post by: Eggbertx on August 13, 2017, 04:04:24 am
Even after deleting everything (There wasn't anything in C:\ProgramData), I'm still getting the unhandled exception error when I run it for the first time.

And it still can't open the configuration manager.
Title: Re: miniSphere 4.8.0
Post by: DaVince on August 13, 2017, 05:58:48 am
Did something change about where miniSphere look for games? On Linux I have all my games in $HOME/Documents/miniSphere/games but it no longer seems to find them.

Edit: scratch that, a second, empty Games folder with a capitalized G appeared alongside it and moving the games into that folder made it work.
Title: Re: miniSphere 4.8.0
Post by: Fat Cerberus on August 13, 2017, 10:31:55 am
Did something change about where miniSphere look for games? On Linux I have all my games in $HOME/Documents/miniSphere/games but it no longer seems to find them.

Edit: scratch that, a second, empty Games folder with a capitalized G appeared alongside it and moving the games into that folder made it work.

Sorry, apparently I messed with the directories while I was refactoring stuff and then forgot about it. :P
Title: Re: miniSphere 4.8.0
Post by: Fat Cerberus on August 13, 2017, 10:58:57 am
Even after deleting everything (There wasn't anything in C:\ProgramData), I'm still getting the unhandled exception error when I run it for the first time.

And it still can't open the configuration manager.

You're right, I get the error for the configuration manager too, when running the bundled Sphere Studio.  My self-compiled version works fine.  There must be something borked in the installer, I'll look into it.

I can't reproduce the error on startup, but it's probably related.  Something's going wrong during plugin loading, it looks like.

edit: Found the problem, there's an outdated SphereStudio.Base.dll in C:\Program Files\miniSphere\ide\Plugins, which conflicts with the real one (this is the main core of the IDE, so it's pretty important).  Delete that and see if that fixes the issue.  I'll fix it in the next release.
Title: Re: miniSphere 4.8.0
Post by: Eggbertx on August 13, 2017, 03:39:48 pm
Alright, that fixed it.
Title: Re: miniSphere 4.8.1
Post by: Fat Cerberus on August 14, 2017, 01:18:45 am
miniSphere 4.8.1 is up to fix a packaging issue for Windows that caused Sphere Studio to not be installed properly.  For anyone that has installed 4.8.0 or earlier, uninstall it, delete everything in C:\Program Files\miniSphere, delete the directory %LOCALAPPDATA%\Spherical, and then install 4.8.1 to ensure a clean slate.
Title: Re: miniSphere 4.8.1
Post by: Fat Cerberus on August 14, 2017, 11:26:08 am
I fixed the SoundStream bug.  It turns out I was calculating the buffer size incorrectly for stereo streams and only half-filling the fragment buffers.  The calculation was only correct for a single channel, which is why I could play stereo files on a mono stream if I doubled the sample rate - most of the time the two channels are very similar so it's just like you're doubling up on samples.

Some technical details on the bug: Allegro al_create_audio_stream() takes a "number of samples" parameter which specifies the size of the buffer:
Code: (c) [Select]
if (!(stream->ptr = al_create_audio_stream(8, 1024, frequency, depth_flag, conf)))
goto on_error;

8 is the number of buffers, 1024 is the number of samples you can put into each buffer.  I misunderstood 1024 to be the overall number of samples, but it's actually the number of samples per channel.  So for mono streams I was fine, but for stereo (or more), the buffer was only getting partially filled and the rest was silence, hence the choppiness and half-speed-ness.

Now if I only I had found the bug before I already released 4.8.1... oh well.
Title: Re: miniSphere 4.8.1
Post by: Eggbertx on August 14, 2017, 11:47:12 am
This isn't a major issue, but Windows 10 blocks the installer from running and requires me to click "run anyway".
Title: Re: miniSphere 4.8.1
Post by: Fat Cerberus on August 14, 2017, 12:17:19 pm
Yeah, the only way around that is to sign the installer with an EV certificate, which cost a fortune.  Without a digital signature Windows sees that it's not commonly downloaded and blocks it.

This the reason most people despise SmartScreen.
Title: Re: miniSphere 4.8.1
Post by: Eggbertx on August 14, 2017, 02:53:29 pm
Does it do this solely because it's an installer?
Title: Re: miniSphere 4.8.1
Post by: Fat Cerberus on August 14, 2017, 03:00:49 pm
Well, because it's an executable file.  For unsigned executables it's based on a popularity check: If very few people have downloaded it, it warns you as a kind of "proceed with caution".  If a bunch of people have downloaded it without it being flagged as a virus then you don't get the warning.

I think the idea is to combat zero-day attacks.

I wouldn't mind the warnings as much if the "Run anyway" button wasn't hidden from view by default.  Firefox and Chrome do the same thing when they block downloads, but at least in that case it's usually based on an actual virus detection rather than a popularity contest. ::)
Title: Re: miniSphere 4.8.1
Post by: Eggbertx on August 14, 2017, 03:04:40 pm
Have you ever had that issue when downloading QtSphere IDE?

Also, I noticed an issue with the new Sphere Studio version. I generally have my project tree on the left side, but whenever I restart Sphere Studio, it resets it back to the right.
Title: Re: miniSphere 4.8.1
Post by: Fat Cerberus on August 14, 2017, 03:07:55 pm
You only get the warning for files that came from the internet.  Locally-sourced files (or if you clear the "came from the internet" attribute) won't get blocked unless Defender actually picks up a virus.

Regarding the project tree: I moved it to the right by default because that's how MSVC is set up in a fresh install.  But I haven't figured out how to get it to save the layout if you customize it :(
Title: Re: miniSphere 4.8.1
Post by: Eggbertx on August 14, 2017, 03:29:24 pm
Ah, alright.
I thought I remember Sphere Studio remembering the location.
Title: Re: miniSphere 4.8.1
Post by: DaVince on August 15, 2017, 09:00:49 am
I wouldn't mind the warnings as much if the "Run anyway" button wasn't hidden from view by default.
I'm sure it's hidden exactly because the whole zero-day security thing wouldn't work if they didn't obscure it, and then it would defeat the whole purpose of the dialog box. Design through common human behavior and all that.
Title: Re: miniSphere 4.8.2
Post by: Fat Cerberus on August 16, 2017, 12:47:38 pm
miniSphere 4.8.2 is up and finally fixes the SoundStream bug.  Yay!
Title: Re: miniSphere 4.8.2
Post by: Rhuan on August 17, 2017, 03:43:52 pm
Not really complicated, just tedious.  I'd basically have to wrap almost all of the SpiderMonkey API in C functions that could be called from miniSphere's C code.  That's part of why I'm trying to isolate the script layer as much as possible from the actual engine code (which has mostly been accomplished, save for the map engine), so that only the files directly related to running JS code need to be updated, stuff like image handling, etc. don't have to be touched.
I was having a look around at details about different JS engines when I was bored earlier today. I suddenly noticed a contender that isn't talked about much, apple's JavaScriptCore:
1. Its compatibility with ES6 is higher than V8 and Spidermonkey: https://kangax.github.io/compat-table/es6/
2. It actually wins the speed tests on are we fast yet: https://arewefastyet.com/
3. It has an available pure C interface: https://karhm.com/JavaScriptCore_C_API/

I found just two downsides:
1. The license is LGPL not BSD/Mozilla - I think this should be ok, from my read of the LGPL you can static link against an LGPL library without importing the LGPL to the rest of your project as long as you make clear that the library is LGPL and provide a link to its license and source.

2. The documentation isn't great... The tutorial I linked above is the best I can find at the moment with some googling.
Title: Re: miniSphere 4.8.2
Post by: Rhuan on August 17, 2017, 07:18:24 pm
Possible bug, though I'm still using 4.7: Shape#draw() (with no parameter) appears to cause a hard crash. (replacing with Shape#draw(screen) removes the crash - but the api documentation says the parameter is optional AND obviously it should throw a missing parameter error not crash if it's not an optional parameter.
Title: Re: miniSphere 4.8.2
Post by: Fat Cerberus on August 17, 2017, 11:15:40 pm
Possible bug, though I'm still using 4.7: Shape#draw() (with no parameter) appears to cause a hard crash.

I can confirm, engine crashes if no surface is specified.  As for whether it's optional, I'm actually a bit confused here.  The documentation says "if surface is omitted, draws to the backbuffer", however the signature of the function is listed as:
Code: [Select]
Shape#draw(surface[, transform]);

Only the second parameter is optional.  I'm thinking the signature is right and the description is wrong, as a surface is required for Model#draw() also.  I could make it optional, but it's currently required for Prim() calls as the first parameter so that'd be kind of awkward.  I'll see.  In any case, I'll definitely fix the crash.

edit: The miniSphere source code is the final authority and it indicates the surface parameter was intended to be optional.  It was just bugged: internally there's a real image_t object representing the backbuffer, but miniSphere tried to draw to NULL if the surface argument is omitted, hence the crash.  It's fixed in the latest build.
Title: Re: miniSphere 4.8.2
Post by: East on August 18, 2017, 10:07:40 am
@Fat Cerberus
Yay an update!
Did you increase the minimum window resolution that makes the engine run in 2x scale windowed mode? My game runs at 480*270 and it was on a small window just now before I updated. I actually like the 2x better anyways.
And on that note, is there any chance we could use filters for windowed mode again sometime, like in the old Sphere 1.x days? I really like the 2x soft-ish look, always reminds me of playing chrono trigger on my emulator.

Also, just updated and getting an error when I try to enter Configuration Manager:
(http://i.imgur.com/UoF78DT.png)

Code: [Select]
See the end of this message for details on invoking 
just-in-time (JIT) debugging instead of this dialog box.

************** Exception Text **************
System.NullReferenceException: Object reference not set to an instance of an object.
   at SphereStudio.Ide.Forms.ConfigManager.UpdatePluginsList()
   at SphereStudio.Ide.Forms.ConfigManager..ctor()
   at SphereStudio.Ide.IdeWindow.menuConfigManager_Click(Object sender, EventArgs e)
   at System.Windows.Forms.ToolStripItem.RaiseEvent(Object key, EventArgs e)
   at System.Windows.Forms.ToolStripMenuItem.OnClick(EventArgs e)
   at System.Windows.Forms.ToolStripItem.HandleClick(EventArgs e)
   at System.Windows.Forms.ToolStripItem.HandleMouseUp(MouseEventArgs e)
   at System.Windows.Forms.ToolStrip.OnMouseUp(MouseEventArgs mea)
   at System.Windows.Forms.ToolStripDropDown.OnMouseUp(MouseEventArgs mea)
   at System.Windows.Forms.Control.WmMouseUp(Message& m, MouseButtons button, Int32 clicks)
   at System.Windows.Forms.Control.WndProc(Message& m)
   at System.Windows.Forms.ToolStrip.WndProc(Message& m)
   at System.Windows.Forms.ToolStripDropDown.WndProc(Message& m)
   at System.Windows.Forms.NativeWindow.Callback(IntPtr hWnd, Int32 msg, IntPtr wparam, IntPtr lparam)


************** Loaded Assemblies **************
mscorlib
    Assembly Version: 4.0.0.0
    Win32 Version: 4.7.2102.0 built by: NET47REL1LAST
    CodeBase: file:///C:/Windows/Microsoft.NET/Framework64/v4.0.30319/mscorlib.dll
----------------------------------------
Sphere Studio
    Assembly Version: 0.0.0.0
    Win32 Version: 0.0.0.0
    CodeBase: file:///C:/Program%20Files/miniSphere/ide/Sphere%20Studio.exe
----------------------------------------
System.Windows.Forms
    Assembly Version: 4.0.0.0
    Win32 Version: 4.7.2104.0 built by: NET47REL1LAST
    CodeBase: file:///C:/WINDOWS/Microsoft.Net/assembly/GAC_MSIL/System.Windows.Forms/v4.0_4.0.0.0__b77a5c561934e089/System.Windows.Forms.dll
----------------------------------------
System
    Assembly Version: 4.0.0.0
    Win32 Version: 4.7.2103.2 built by: NET47REL1LAST
    CodeBase: file:///C:/WINDOWS/Microsoft.Net/assembly/GAC_MSIL/System/v4.0_4.0.0.0__b77a5c561934e089/System.dll
----------------------------------------
System.Drawing
    Assembly Version: 4.0.0.0
    Win32 Version: 4.7.2046.0 built by: NET47REL1
    CodeBase: file:///C:/WINDOWS/Microsoft.Net/assembly/GAC_MSIL/System.Drawing/v4.0_4.0.0.0__b03f5f7f11d50a3a/System.Drawing.dll
----------------------------------------
SphereStudio.Base
    Assembly Version: 0.0.0.0
    Win32 Version: 0.0.0.0
    CodeBase: file:///C:/Program%20Files/miniSphere/ide/SphereStudio.Base.DLL
----------------------------------------
System.Core
    Assembly Version: 4.0.0.0
    Win32 Version: 4.7.2102.0 built by: NET47REL1LAST
    CodeBase: file:///C:/WINDOWS/Microsoft.Net/assembly/GAC_MSIL/System.Core/v4.0_4.0.0.0__b77a5c561934e089/System.Core.dll
----------------------------------------
System.Configuration
    Assembly Version: 4.0.0.0
    Win32 Version: 4.7.2046.0 built by: NET47REL1
    CodeBase: file:///C:/WINDOWS/Microsoft.Net/assembly/GAC_MSIL/System.Configuration/v4.0_4.0.0.0__b03f5f7f11d50a3a/System.Configuration.dll
----------------------------------------
System.Xml
    Assembly Version: 4.0.0.0
    Win32 Version: 4.7.2102.0 built by: NET47REL1LAST
    CodeBase: file:///C:/WINDOWS/Microsoft.Net/assembly/GAC_MSIL/System.Xml/v4.0_4.0.0.0__b77a5c561934e089/System.Xml.dll
----------------------------------------
WeifenLuo.WinFormsUI.Docking
    Assembly Version: 2.16.0.0
    Win32 Version: 2.16.0.0
    CodeBase: file:///C:/Program%20Files/miniSphere/ide/WeifenLuo.WinFormsUI.Docking.DLL
----------------------------------------
AudioPlayerPlugin
    Assembly Version: 1.0.0.0
    Win32 Version: 1.0.0.0
    CodeBase: file:///C:/Program%20Files/miniSphere/ide/Plugins/AudioPlayerPlugin.dll
----------------------------------------
ClassicSpherePlugin
    Assembly Version: 1.0.0.0
    Win32 Version: 1.0.0.0
    CodeBase: file:///C:/Program%20Files/miniSphere/ide/Plugins/ClassicSpherePlugin.dll
----------------------------------------
Sphere.Plugins
    Assembly Version: 1.2.1.0
    Win32 Version: 1.2.1
    CodeBase: file:///C:/Program%20Files/miniSphere/ide/Sphere.Plugins.DLL
----------------------------------------
FontEditPlugin
    Assembly Version: 1.0.0.0
    Win32 Version: 1.0.0.0
    CodeBase: file:///C:/Program%20Files/miniSphere/ide/Plugins/FontEditPlugin.dll
----------------------------------------
ImageEditPlugin
    Assembly Version: 1.0.0.0
    Win32 Version: 1.0.0.0
    CodeBase: file:///C:/Program%20Files/miniSphere/ide/Plugins/ImageEditPlugin.dll
----------------------------------------
LegacySpherePlugin
    Assembly Version: 1.0.0.0
    Win32 Version: 1.0.0.0
    CodeBase: file:///C:/Program%20Files/miniSphere/ide/Plugins/LegacySpherePlugin.dll
----------------------------------------
MapEditPlugin
    Assembly Version: 1.0.0.0
    Win32 Version: 1.0.0.0
    CodeBase: file:///C:/Program%20Files/miniSphere/ide/Plugins/MapEditPlugin.dll
----------------------------------------
miniSphereGdkPlugin
    Assembly Version: 4.8.2.2246
    Win32 Version: 4.8.2.2246
    CodeBase: file:///C:/Program%20Files/miniSphere/ide/Plugins/miniSphereGdkPlugin.dll
----------------------------------------
ScriptEditPlugin
    Assembly Version: 1.0.0.0
    Win32 Version: 1.0.0.0

Edit:
When I try to test-run my new projects (as in, games I didn't migrate from Sphere 1.x), I get this error:
Code: [Select]
setting up Cellscript environment...
evaluating 'Cellscript.mjs'...
   E: ReferenceError: identifier 'describe' undefined
   @ [Cellscript.mjs:13]
1 error(s), 0 warning(s).
================== Failed to build: Legend ==================

These are lines 13 to 23 in Cellscript.mjs (I haven't changed anything in them after the update):
Code: [Select]

// describe the game we're building.  this gets written to the game manifest
// (game.json) at the end of the build.

describe("Legend", {
version: 2,  // Sphere v2
author: "East",
summary: "",
resolution: '400x300',
main: 'scripts/main.js',
});
Title: Re: miniSphere 4.8.2
Post by: Fat Cerberus on August 18, 2017, 10:17:53 am
Uninstall, and reinstall 4.8.2.  There was a bug in the original 4.8.0 installer that installed a DLL in the wrong place and screwed everything up, and Inno Setup won't remove misplaced files if you just do an upgrade.  You'll need to do a full uninstall to get rid of the old cruft.

Also go into %LOCALAPPDATA% and delete "Spherical" and "Sphere_Engine_Group" folders to ensure you have a clean slate.

As for the scaling, I changed the scaling algorithm in 4.8.  Previously it would do a 2X scale if the game was 400x300 or less.  The new algorithm actually compares the game resolution to the real screen resolution and scales accordingly.  I did that because low-res games ended up being really tiny on HiDPI screens otherwise (think 4K monitor, 3840x2160).
Title: Re: miniSphere 4.8.2
Post by: Fat Cerberus on August 18, 2017, 10:25:55 am
The Cell API changed a bit a few versions ago, I'm not sure exactly how much changed though since I don't know which version you had before.  Here's the current template, sans comments:

Code: [Select]
import { transpile } from 'cell-runtime';

Object.assign(Sphere.Game, {
name: "{0}",
author: "{1}",
summary: "{2}",
resolution: '{3}x{4}',
main: '@/bin/main.js',
});

transpile('@/bin', files('src/*.mjs'));
transpile('@/bin', files('src/*.js'));

install('@/images', files('images/*.png', true));
install('@/music',  files('music/*.ogg', true));
install('@/',       files('icon.png'));

That should hopefully let you see what needs to be changed to get it working again.
Title: Re: miniSphere 4.8.2
Post by: East on August 18, 2017, 10:34:38 am
Everything works perfect now. Thanks :)
Title: Re: miniSphere 4.8.3
Post by: Fat Cerberus on August 19, 2017, 02:46:05 am
miniSphere 4.8.3 is up with a bunch of bug fixes (including two engine-crash bugs), under-the-hood improvements, and a new function, FS.relativePath():
https://github.com/fatcerberus/minisphere/blob/v4.8.3/docs/sphere2-core-api.txt#L555-L560
Title: Re: miniSphere 4.8.3
Post by: Rhuan on August 19, 2017, 05:48:13 am
What are your thoughts on adding typescript support to cell?

So far I've stuck with writing ES5 javascript and not using a transpiler but I've noticed I'm frequently checking my type consistency and having built in type checking would probably save me quite a bit of time.

**I note that I've just read back and found that you had a negative of few of Typescript's type system a few months back, could it be in cell as an option for people who want it?
Title: Re: miniSphere 4.8.3
Post by: Rhuan on August 19, 2017, 08:59:19 am
I've found another bug:
If you create an object with a function but omit a parameter it needs miniSphere hard crashes if you try and access that field later
e.g.:
Code: [Select]
function CoolObject(a, b, c)
{
  this.a = a;
  this.b = b;
  this.c = c;
}

var fun = new CoolObject(5, 6);

throw fun.c;//results in a hard crash

Obviously this is bad code that would always cause problems - but ideally it would trigger miniSPhere's error handling to make it easier to debug OR it should give you a "undefined" value rather than

Code: [Select]
Crashed Thread:        9

Exception Type:        EXC_CRASH (SIGABRT)
Exception Codes:       0x0000000000000000, 0x0000000000000000
Exception Note:        EXC_CORPSE_NOTIFY

Thread 9 Crashed:
0   libsystem_kernel.dylib        0x00007fff9a1bbd42 __pthread_kill + 10
1   libsystem_pthread.dylib       0x00007fff9a2a9457 pthread_kill + 90
2   libsystem_c.dylib             0x00007fff9a121420 abort + 129
3   spheredev.miniSphere          0x0000000106bb2bfc duk_default_fatal_handler + 9 (duk_api_heap.c:68)
4   spheredev.miniSphere          0x0000000106bbc027 duk_fatal_raw + 15 (duk_api_stack.c:5428)
5   spheredev.miniSphere          0x0000000106bbc00e duk_err_longjmp + 102 (duk_error_longjmp.c:102)
6   spheredev.miniSphere          0x0000000106bbdd23 duk_err_create_and_throw + 300 (duk_api_bytecode.c:48)
7   spheredev.miniSphere          0x0000000106bb0e96 duk_err_handle_error_fmt + 187 (duk_api_stack.c:5801)
8   spheredev.miniSphere          0x0000000106bb45ed duk_hobject_getprop + 2391 (duk_api_stack.c:3886)
9   spheredev.miniSphere          0x0000000106bb3c7e duk_get_prop + 45 (duk_api_object.c:30)
10  spheredev.miniSphere          0x0000000106bf742a _al_mangled_main + 2922 (main.c:316)
11  spheredev.miniSphere          0x0000000106d7b8da +[AllegroAppDelegate app_main:] + 23
12  com.apple.Foundation          0x00007fff863308ad __NSThread__start__ + 1243
13  libsystem_pthread.dylib       0x00007fff9a2a693b _pthread_body + 180
14  libsystem_pthread.dylib       0x00007fff9a2a6887 _pthread_start + 286
15  libsystem_pthread.dylib       0x00007fff9a2a608d thread_start + 13
Title: Re: miniSphere 4.8.3
Post by: Rhuan on August 19, 2017, 09:23:05 am
And another one I've seen a few times but I don't think I've mentioned before.

If I attempt to access a negative indexed property of an array, e.g.
Code: [Select]
var arr = [0,1,2,5];
throw arr[-1];

I get a bizarrely distorted error handling screen, see attachment.

EDIT: trying to get ~1000 sprites to collide with each other and hitting error after error :(
Title: Re: miniSphere 4.8.3
Post by: Fat Cerberus on August 19, 2017, 10:12:50 am
The root cause of the segfault is:

Code: [Select]
throw undefined;  // CRASH! *BOOM* :pig_nose: *MUNCH*

You always find the weirdest crashes.  It's not the access itself that crashes it, it's that you're throwing undefined instead of an exception object and I apparently didn't account for that possibility.  If I change your first example instead to:

Code: [Select]
function CoolObject(a, b, c)
{
  this.a = a;
  this.b = b;
  this.c = c;
}

var fun = new CoolObject(5, 6);

SSj.log(fun.c);

Then it works fine.

As for the distorted error screen, did you happen to have a transformation matrix active at the time?  If so, I fixed that in 4.8.3.
Title: Re: miniSphere 4.8.3
Post by: Rhuan on August 19, 2017, 10:29:13 am
Thanks for looking at these.

A couple more notes, for the first issue, I get the same effect (that is a segfault) if I do this:
Code: [Select]
function CoolObject(a, b, c)
{
  this.a = a;
  this.b = b;
  this.c = c;
}

var fun = new CoolObject(5, 6);
var bob = [];
bob[fun.c];
i.e. I try and use that undefined property as a reference.


For the second issue, I don't believe there was a screen transformation set, also I can't reproduce it now, so I'll revert to you next time I see it.
Title: Re: miniSphere 4.8.3
Post by: Fat Cerberus on August 19, 2017, 10:50:27 am
It also crashes if you throw null;.  I tested throw with a bunch of stuff originally--numbers, objects, strings, etc.  It works for everything except null and undefined, because the uncaught error handler tries to read a filename and line number from the thrown value and those are the only two values that are not object coercible.  So the property read throws ANOTHER error, which causes Duktape to panic and call abort().  I'm fixing it as we speak.

I couldn't reproduce any crash with bob[fun.c]; though.  I pasted that at the top of my script and nothing happened, it just proceeded as usual.
Title: Re: miniSphere 4.8.3
Post by: Rhuan on August 19, 2017, 11:06:38 am
I couldn't reproduce any crash with bob[fun.c]; though.  I pasted that at the top of my script and nothing happened, it just proceeded as usual.
That's weird, I also now can't reproduce the error so it must have been something else, unfortunately my first segfault was with no "throw"s anywhere, I added throws to try and catch it, so there definitely is another error but now I don't know what it actually was :(
Title: Re: miniSphere 4.8.3
Post by: Rhuan on August 19, 2017, 09:07:53 pm
https://github.com/fatcerberus/minisphere/issues/153

I've spent some time staring at JS engines recently, looking at compatibility, performance, and usability.

In addition to Spidermonkey (SM) and V8 I've also looked at Apple's Javascriptcore (JSC_ and I've taken a quick look at Microsoft's Chakracore (CC).

As far as I can see none of them run TypeScript natively (suggestion from your issue log above) that is unless I'm missing something the TypeScript benchmark some of them show is the speed of compiling typescript with the javascript typescript compiler.

Performance
Different ones are better for different things. V8 seems to get the best results out of javascript that's written for readability not speed, whilst javascriptcore gets the worst results from such code. Spidermonkey comes out in the middle. flipping to code that has been written for speed and uses optimisation tricks javascriptcore jumps to the lead, outperforming V8 and spidermonkey by significant margins in some cases. Microsoft's ChakraCore seems to generally get the worst results on most benchmarks though there are a few outlying positives. I note that even bad results on the benchmarks = 10 * faster than miniSphere's current performance; the good results on the benchmarks are 70 * faster or more.

I note that from the tests I did it looks like far more effort has gone into optimising V8 and JSC performance than CC and SM, presumably as JSC is now used in a lot of IOS games and V* is used in node.js and probably android apps.

Compatibility with ES6
They're all pretty good JSC gets 99%, SM and v8 both get 97% and CC gets 96. http://kangax.github.io/compat-table/es6/

Garbage Collection
CC: uses parallel processing to have garbage collection always happening rather than pausing for it
V8: incremental garbage collection but stops code execution when doing it
SM: no idea what it does - only documentation on GC in SM I can find says at the top that it (the documentation) is out of date and wrong
JSC: uses parallel processing to avoid stopping main thread for GC


APIs:
JSC and CC both appear to have pure C apis available. (JSC also has an objective C api but it's a wrapper around the C api and can be safely ignored).
V8 and Spidermonkey both have C++ apis as the only option.

Documentation:
V8: good documentation
CC: good documentation
SM: mixed documentation - wiki with loads of out of date content, though the key information for how to use it is there
JSC: mixed documentation - lots of focus on the objective C wrapper api rather than C, also most articles assume that any developer using it is targeting IOS

Licenses:
V8: BSD
CC: MIT
SM: MPL2
JSC: LGPL
All of these seem ok to me as long as the source is kept separate from the miniSphere source.

Conclusion
Considering the potential to avoid needing a C wrapper around C++ I think that CC or JSC would be nicer options than V8 or SM, then based on benchmark performance I think that JSC just wins.

Anyway comparing these was a nice bit of fun, now I need to finish my collision detection...
Title: Re: miniSphere 4.8.3
Post by: Fat Cerberus on August 20, 2017, 12:09:14 am
Documentation:
V8: good documentation
CC: good documentation
SM: mixed documentation - wiki with loads of out of date content, though the key information for how to use it is there
JSC: mixed documentation - lots of focus on the objective C wrapper api rather than C, also most articles assume that any developer using it is targeting IOS

That's what I found with SM, too, the wiki is not very well maintained at all.  For example, the release notes for SM 45, emphasis is mine:
https://developer.mozilla.org/en-US/docs/Mozilla/Projects/SpiderMonkey/Releases/45

Quote
This page is not complete.

These release notes are an incomplete draft and will remain so well after SpiderMonkey 45 is released.

[...]

-- Apr 14, 2016

That, um, doesn't exactly inspire confidence.  Not to mention the fact that SpiderMonkey 52 is still listed as a "future release" and doesn't even have a wiki page yet--despite the fact that Firefox is already up to v55.  Out-of-date documentation is worse than no documentation at all.  I honestly get the sense that Mozilla really doesn't care about embedders anymore.\

It *does* surprise me that you say V8 has good documentation, since I think I remember FJ saying their documentation was very poor.  Even so, from what I've heard people say, V8's garbage collector is apparently very lazy and almost never runs at all, which isn't ideal when you have native resources tied to the lifetime of a JS object.  This is one nice thing about Duktape's GC: it's refcounted backed by a stop-the-world mark-and-sweep collector, so barring a circular reference, things tend to get finalized the minute they go out of scope, which is nice.

The one that interests me most is ChakraCore.  MS's documentation has always been second-to-none, and it's MIT-licensed which would be a great fit for miniSphere.  I can navigate around the LGPL, but I'd prefer not to have to deal with it.  And I don't really want to have to deal with spotty documentation if I can avoid it, either.

Re: TypeScript: I don't really expect any JS engine to support it natively, that's not what I meant by the TypeScript comment.  It's just that, right now I'm using TS as a gloried ES6 transpiler because Duktape (remember, Cell uses Duktape too) is way too slow to use the full type system.  I tried it, it was ridiculously slow.  And type-checking requires all sources to be compiled together, which made it even worse.
Title: Re: miniSphere 4.8.3
Post by: Fat Cerberus on August 20, 2017, 03:36:53 pm
I'm experimenting with ChakraCore, the pure C API is really nice and doesn't seem overly bloated.  I'm trying to see if I can get it to run some code passed in as a string and return the value it returns to C.  Then I can see about getting some test bindings going.

A big mistake I made with duktape was, because it was so easy to get a functioning engine off the ground with, I just used it directly without wrapping it with any kind of abstraction (especially since the stack-based API makes a generic abstraction difficult).  Not an exaggeration, I had my Spectacles battle engine running within weeks of starting development:
http://forums.spheredev.org/index.php/topic,1215.msg5428.html#msg5428

One thing that's inconvenient that I also had to deal with when I experimented with V8 is that the API expects strings to be widechar (`wchar_t`), but miniSphere uses UTF-8 internally.  Luckily there's a function to create string values from a UTF-8 string which I can get the `wchar_t*` for.  So it's an extra step but isn't too much of a hassle in practice, especially since I can abstract around it.
Title: Re: miniSphere 4.8.3
Post by: Rhuan on August 20, 2017, 04:28:39 pm
I've had a quick read and it seems relatively simple to work with, I look forward to seeing how you get on, I'd love to have my sprite update code and collision code run with a JIT - performance is ok under duktape but it's a struggle to keep a steady 60 FPS whilst handling 100+ entities now I've added collisions.

I'd still also like to try a JSC version of sphere at some point, maybe when you're done with CC I'll see if I can swap it to JSC in a test build; from reading both APIs they seem functionally quite similar - same sorts of calls to make just different names for them, tbh I don't know which will actually be better - CC has far less test results available for it as it's so much newer and the only browser that uses it is windows only.

A couple of notes on my points above:
V8 documentation: a lot of content on their wiki has been written this year, so it may have been a lot worse before.

General state of the different engines: V8, JSC and CC have all been undergoing rapid development over the last year for different reasons with much effort spent optimising each and even replacing core elements of their architecture (e.g. JSC got a whole new GC system in January, CC only got non-windows support a year ago and non-windows JIT support in October)
SM on the other hand seems to be slowly dying unless I'm missing something.
Title: Re: miniSphere 4.8.3
Post by: Fat Cerberus on August 20, 2017, 06:56:53 pm
CC apparently supports ES6 modules (.mjs) natively which is really nice to see:
https://github.com/Microsoft/ChakraCore/wiki/Roadmap

ES6 module syntax and semantics are much more elegant than CommonJS (which let's face it, CJS is a hack).  Plus they run in permanent strict-mode which helps catch bugs.
Title: Re: miniSphere 4.8.3
Post by: Rhuan on August 20, 2017, 07:07:52 pm
CC on the whole does look very nice my only concerns with it were that:
a) it's a bit new and less thoroughly tested than the alternatives
b) it's unlikely to be the fastest (though they're working on that)
c) I couldn't benchmark it myself without building some as microsoft edge can't run on a mac
d) Any new low level features are added to windows first and take time to be ported to other platforms
Title: Re: miniSphere 4.8.3
Post by: Fat Cerberus on August 21, 2017, 12:54:06 pm
One thing that's going to make this more challenging is, regardless of which JS engine I pick, exception handling differs from Duktape.  In Duktape, if an exception is thrown, it propogates via longjmp() all the way to the nearest catch or duk_pcall(), even cutting through C functions in the process.  CC, SM and probably V8 will throw an exception through JS code, but stop at a JS-C boundary where the error must be detected and handled manually, e.g. by rethrowing and returning from the C function.

Essentially in Duktape, any C code executed during script processing is just an extension of the normal JS environment.  This is very convenient, and there are a few spots in the miniSphere codebase that rely being able to immediately interrupt a C callback with a JS exception from any stack depth, e.g. argument validation:
https://github.com/fatcerberus/minisphere/blob/v4.8.3/src/minisphere/utility.c#L287

So those would need to be refactored, unless I could come up with some way to emulate it.
Title: Re: miniSphere 4.8.3
Post by: Rhuan on August 21, 2017, 01:31:27 pm
Could you simulate this error handling by doing the following

1. within the function that wants to generate an error the JsCreateError(message, out_error) function
2. an early return (to send sphere back into the JS)
3. wrap all JS execution commands in an error catching function that displays the message

PS I've built ChakraCore on MacOs and started doing some speed/functionality tests - it's close to parity with V8 and JSC for performance which is very nice to see, it seems to have a slightly higher threshold for triggering it's optimising JIT than JSC has which means you have to get to higher levels of recursion before its top speeds come out but those top speeds are pretty crazy.
Title: Re: miniSphere 4.8.3
Post by: Fat Cerberus on August 21, 2017, 03:20:39 pm
So far I've written a bunch of functions to abstract around the JS engine and fit in with the style of the rest of the codebase:
https://github.com/fatcerberus/minisphere/blob/chakra-js/src/shared/ecmunch.c

I've done some testing with it and I can successfully set object properties, create functions, eval code and get the result back, etc.  I haven't figured out how to get it to load a module yet, though.
Title: Re: miniSphere 4.8.3
Post by: Fat Cerberus on August 21, 2017, 03:38:29 pm
Something interesting about CC is that any JsValueRefs stored as local variables on the stack are apparently protected from garbage collection without needing to explicitly root them nor call JsRelease().  I'm not really sure how that works: Does the GC actually examine the native C/C++stack frames looking for refs?

Although I guess you'd need to have a system like that if you're doing GC off-thread.  Otherwise the object might get collected before you had a chance to root it.
Title: Re: miniSphere 4.8.3
Post by: Rhuan on August 21, 2017, 04:51:57 pm
So far I've written a bunch of functions to abstract around the JS engine and fit in with the style of the rest of the codebase:
https://github.com/fatcerberus/minisphere/blob/chakra-js/src/shared/ecmunch.c

I've done some testing with it and I can successfully set object properties, create functions, eval code and get the result back, etc.  I haven't figured out how to get it to load a module yet, though.
Hmm I said the documentation was good... But it's not all in place, for some reason setting up modules is documented in ChakraCore.h but not in the wiki.
For executing a module looks like you need:
JsInitializeModuleRecord
JsParseModuleSource
and
JsModuleEvaluation

As far as I can tell these are all ES6 style, not commonJS though I haven't gone into the detail to check that.

I assume commonJS may need to be added similar to other engines.

Something interesting about CC is that any JsValueRefs stored as local variables on the stack are apparently protected from garbage collection without needing to explicitly root them nor call JsRelease().  I'm not really sure how that works: Does the GC actually examine the native C/C++stack frames looking for refs?

Although I guess you'd need to have a system like that if you're doing GC off-thread.  Otherwise the object might get collected before you had a chance to root it.

A quick look gave me the following:
Quote
A Chakra runtime will automatically track JsRef references as long as they are stored in local variables or in parameters (i.e. on the stack). Storing a JsRef somewhere other than on the stack requires calling JsAddRef and JsRelease to manage the lifetime of the object, otherwise the garbage collector may free the object while it is still in use.
Title: Re: miniSphere 4.8.3
Post by: Fat Cerberus on August 21, 2017, 06:48:52 pm
Maybe the module functions are experimental/not released yet?  I'll check into that later.
Title: Re: miniSphere 4.8.3
Post by: Flying Jester on August 21, 2017, 09:40:52 pm
A couple of notes on my points above:
V8 documentation: a lot of content on their wiki has been written this year, so it may have been a lot worse before.

It was non-existent before. A big part of when I switched to SM for TurboSphere was that V8 was largely undocumented, and what was there was unusable. The API changed wildly every couple months (compared with SM's, which is quite stable in comparison), and there was no documentation at all, except three pages of outdated descriptions of APIs that no longer existed, and two examples that were not updated since before I began the project.

Regarding API stability, unless things have drastically changed in the last couple months since I updated some Turbo code to SM 52, SM is still way, way more stable than V8.

I also caution against using V8 for anything relating to graphics or files, since its GC is very unreliable. SM's was much more stable, always releasing memory in a timely manner. This made V8 basically unusable for Galileo or any FS work.
Title: Re: miniSphere 4.8.4
Post by: Fat Cerberus on August 22, 2017, 02:03:56 am
miniSphere 4.8.4 is out now with several bug fixes and improvements.  No new APIs this time.
Title: Re: miniSphere 4.8.4
Post by: Fat Cerberus on August 22, 2017, 11:06:22 am
Just going to say how awesome it is every time I get to see this. ;D
Title: Re: miniSphere 4.8.4
Post by: Fat Cerberus on August 22, 2017, 01:34:29 pm
I got ChakraCore up and running! :D

I set up the following test:
Code: (C) [Select]
static js_value_t*
js_chakraTest(js_value_t* this, int num_args, js_value_t* args[], bool is_ctor)
{
    printf("Something stupid happened and then you got eaten by a %s!\n",
        js_value_as_string(args[0]));
    return NULL;
}

js_value_t* function = js_value_new_function("chakraTest", js_chakraTest, 0);
js_value_set(js_global_object(), "chakraTest", function);
js_value_unref(function);
js_value_unref(js_value_new_eval(lstr_new("chakraTest('pig');")));

Output:
Code: [Select]
Something stupid happened and then you got eaten by a pig!

The js_value_unref()s are an unfortunate side effect of the abstraction--I'm not longer passing around the JsRefs directly, so I need to manage the object lifetime manually. :(
Title: Re: miniSphere 4.8.4
Post by: Rhuan on August 22, 2017, 01:54:43 pm
Quote from: Fat Cerberus link=msg=9964
The js_value_unref()s are an unfortunate side effect of the abstraction--I'm not longer passing around the JsRefs directly, so I need to manage the object lifetime manually. :(
Hmm, will that cause a problem with garbage collection? I can see it getting annoying at least, anyway you could add at least some of them to the abstraction layer?

I assumed you'd have got a function callback set up sooner I guess you focussed on the abstraction layer first?

Good work nonetheless.

(I wrote an equivalent test for jsc on Saturday but without any abstraction, which meant it was much quicker to do but ultimately much less useful I suppose)
Title: Re: miniSphere 4.8.4
Post by: Fat Cerberus on August 22, 2017, 02:06:54 pm
Quote from: Fat Cerberus link=msg=9964
The js_value_unref()s are an unfortunate side effect of the abstraction--I'm not longer passing around the JsRefs directly, so I need to manage the object lifetime manually. :(
Hmm, will that cause a problem with garbage collection? I can see it getting annoying at least, anyway you could add at least some of them to the abstraction layer?

I assumed you'd have got a function callback set up sooner I guess you focussed on the abstraction layer first?

Good work nonetheless.

(I wrote an equivalent test for jsc on Saturday but without any abstraction)

Regarding the abstraction layer: I wanted to get it out of the way first, to avoid making the same mistake I made with Duktape where there was no abstraction to begin with and implementing the extra layer now would be prohibitively time-consuming relative to the benefits.  It's bad enough I'm going to have to refactor everything in pegasus.c and vanilla.c (and probably some stuff in map_engine.c too).  The v1 and v2 APIs taken together account for about 500-600 functions, methods and properties, the only saving grace is going to be that those are mostly all stubs with the actual work done deeper in the engine.

Writing an abstraction layer first has actually become my M.O. of late.  It makes it easier for me writing the higher-level code because I tend to change low-level details around a lot during early development of big features.  If I can minimize having to constantly rewrite outside call sites in the process, it speeds things along in the long run.

Garbage collection shouldn't be negatively impacted as long as I don't forget to free values.  In practice I don't see needing to create that many JS objects in C code in normal operation, so it shouldn't be too annoying.  I could probably do some of the work in the abstraction layer, e.g. by calling `js_value_unref()` after setting a property (since it's unlikely the value is still needed on the native side after that).
Title: Re: miniSphere 4.8.4
Post by: Fat Cerberus on August 22, 2017, 02:19:54 pm
Another advantage to writing the abstraction first: It allowed me to quickly get familiar with a large swath of the API, rather than only a single focused portion of it as would have been the case if I just tried to set up a callback right away.
Title: Re: miniSphere 4.8.4
Post by: Rhuan on August 22, 2017, 02:46:29 pm
Regarding the abstraction layer: I wanted to get it out of the way first, to avoid making the same mistake I made with Duktape where there was no abstraction to begin with and implementing the extra layer now would be prohibitively time-consuming relative to the benefits.  It's bad enough I'm going to have to refactor everything in pegasus.c and vanilla.c (and probably some stuff in map_engine.c too).  The v1 and v2 APIs taken together account for about 500-600 functions, methods and properties, the only saving grace is going to be that those are mostly all stubs with the actual work done deeper in the engine.
I could help with the refactoring if you'd like, once you've got the basic framework set up and done a few functions I'm sure I'd be able to match the style and basic methodology. Let me know if you want to split it.
Title: Re: miniSphere 4.8.4
Post by: Eggbertx on August 22, 2017, 06:14:03 pm
Oh boy, another library that I'll have to get working when/if I ever get around to getting more done on Android/emscripten support :P
Really though, getting the Javascript library (whatever that is/will be) is the least of my worriest. I've yet to get Allegro to work in either emscripten or Android Studio.
Title: Re: miniSphere 4.8.4
Post by: Fat Cerberus on August 22, 2017, 06:16:04 pm
If you ever get miniSphere to work on Android, let me know!  Allegro is supposed to support it but I couldn't figure out their build systems.
Title: Re: miniSphere 4.8.4
Post by: Rhuan on August 22, 2017, 06:35:18 pm
ChakraCore on Android would be a project... BUT: https://github.com/Microsoft/ChakraCore/issues/3179 some people apparently made an unofficial port of chakracore (and node.js running on top of chakra) to ios so it can almost certainly be done to android.

I don't think emscripten would be the way to do it though, CC is big.

(Regarding the node.js comment - there's a shim written by microsoft to translate the chakracore run time into the V8 runtime so people can link node.js to chakracore + the shim instead of V8)

If you ever get miniSphere to work on Android, let me know!  Allegro is supposed to support it but I couldn't figure out their build systems.
I couldn't get Allegro's MacOs build systems working well so i wrote my own xcodeproject/makefile for it in the end when I was getting minisphere to compile. I daren't think what their android system is like.
Title: Re: miniSphere 4.8.4
Post by: Eggbertx on August 22, 2017, 06:36:42 pm
Will do. And looking around, it looks like without making some serious changes and compromises, getting Sphere on the web would be almost impossible. It would take significantly less effort to port a sphere game to HTML5.

Come to think of it though, porting the 1.x (for starters) API might not be such a bad idea. Porting the threading stuff could be a bit difficult, and game developers might have to deal with cross-browser compatibility, but it would still likely be easier than porting all of miniSphere to JavaScript.

Re. ChakraCore, has it been officially decided that miniSphere will use that?
Title: Re: miniSphere 4.8.4
Post by: Rhuan on August 22, 2017, 06:48:46 pm
Couldn't Sphere run on the web using an individual's browsers JS engine + an implementation of the API in HTML 5? Yes that would require almost a line 1 re-write of the engine but on the face of it I think that would result in better performance and a more sensible system than trying to port the existing codebase.

@Fat Cerberus: what was your plan with Oozaru?

It seems to me that a web implementation and a desktop implementation should be different rather than one being a direct port of the other.
Title: Re: miniSphere 4.8.4
Post by: Fat Cerberus on August 22, 2017, 06:51:05 pm
Couldn't Sphere run on the web using an individual's browsers JS engine + an implementation of the API in HTML 5? Yes that would require almost a line 1 re-write of the engine but on the face of it I think that would result in better performance and a more sensible system than trying to port the existing codebase.

Yes, this is exactly what Oozaru is meant to be.  I've just be preoccupied with miniSphere since I'm trying to finalize the v2 API before I write an entire second implementation of it.
Title: Re: miniSphere 4.8.4
Post by: Eggbertx on August 22, 2017, 06:59:08 pm
Oh, I didn't realize Oozaru existed, I was just about to start messing with my own implementation, though I was going to start with the 1.x API and work my way up.
Title: Re: miniSphere 4.8.4
Post by: Rhuan on August 22, 2017, 07:20:39 pm
Oh, I didn't realize Oozaru existed, I was just about to start messing with my own implementation, though I was going to start with the 1.x API and work my way up.
1.x was where it all started and I loved it for a while - Sphere 1.1 (at the time) is what I learnt to programme with.
But the v2 API is great too, I'm not intending to do anything around v1 anymore (as long as Fat Cerberus keeps fixing the bugs I find in v2 stuff :P)
Title: Re: miniSphere 4.8.4
Post by: Eggbertx on August 22, 2017, 07:51:47 pm
The only reason I'd go with 1.x compatibility is because it's simpler, and because miniSphere still supports it. And there are old games that require it and I don't want to break those.
Title: Re: miniSphere 4.8.4
Post by: Fat Cerberus on August 22, 2017, 07:58:07 pm
The way the Sphere v1 API is set up is actually not very browser-friendly at all.  There is no concept of an event loop in Sphere 1.x so games necessarily do this:

Code: (javascript) [Select]
while (true) {
    /* update stuff */

    /* render stuff */

    FlipScreen();
}

And of course MapEngine() itself is blocking.  Do anything like that on a webpage, and you lock up the browser/get your script forcibly terminated for your troubles.  To my knowledge there's no function like miniSphere's Sphere.run() that you can call in a browser to pump the event loop, you need to let your script return naturally.
Title: Re: miniSphere 4.8.4
Post by: Eggbertx on August 22, 2017, 08:00:06 pm
Ah, damn. If only Oracle hadn't run Java applets into the ground and then ditched them :/
Title: Re: miniSphere 4.8.4
Post by: Eggbertx on August 23, 2017, 04:04:43 am
I'm starting to seriously work on porting miniSphere to Android. After a lot of frustration, I'm finally starting to get Allegro to work. I already got duktape to work a while ago, since that was obviously pretty easy.

https://joelme.twilightparadox.com/s/tlh9TFLdNPHvwZ6
Title: Re: miniSphere 4.8.4
Post by: Fat Cerberus on August 23, 2017, 11:11:17 am
Re. ChakraCore, has it been officially decided that miniSphere will use that?

It's not technically set in stone yet, but it's looking like the best option.  V8 and SpiderMonkey are both non-starters because they have C++ APIs which rely heavily on C++ block scoping and RAII to manage the scope of objects.  A good chunk of miniSphere's code would have to be rewritten to accommodate and I really don't want to compromise the nice C purity the project has now.  Plus I hate C++ with a burning passion O:)

JavaScriptCore is still on the table, but the LGPL license makes it less than ideal.  It's a secondary option if for some reason Chakra doesn't work out.
Title: Re: miniSphere 4.8.4
Post by: Eggbertx on August 23, 2017, 12:22:31 pm
Sounds good. I just want to not have to worry about rewriting that part of the android port. Allegro isn't fully figured out yet, but I've got graphics mostly down, shaders and advanced stuff aside, and I'm starting to work on porting libmng as well, since that looks simple enough.

I was thinking about it, and unfortunately, a lot of Sphere games will have to be majorly written, or they may be impossible to port because of the inherent input limitations in mobile devices.
I don't know how I'm going to do it right now, but when I get it fully working, I'm goiNg to figure out how to cleanly merge this into the mainline codebase.
Title: Re: miniSphere 4.8.4
Post by: Rhuan on August 23, 2017, 12:33:15 pm
I had in my mind the idea that ChakraCore could be selected as a compile time CC or duktape depending what you're building for. duktape gives you a smaller more portable build that uses less resources, whereas CC gives you ES6 and higher performance - games using ES6 or even written in typescript could be compiled using CC and typescript into ES5 single file JS for duktape execution in "smaller" environments.

Not sure if this is what Fat Cerberus is planning though (seems a shame to bin the work done implementing duktape, particularly as it is so much lighter and more portable).

That said assuming such a model was established I was hoping to later add code to make JSC a 3rd option, partly because I want to compare how sphere runs with JSC vs CC and partly because JSC is included in IOS - it would make an IOS port very easy.
Title: Re: miniSphere 4.8.4
Post by: Fat Cerberus on August 23, 2017, 12:37:52 pm
I had in my mind the idea that ChakraCore could be selected as a compile time CC or duktape depending what you're building for. duktape gives you a smaller more portable build that uses less resources, whereas CC gives you ES6 and higher performance.

Not sure if this is what Fat Cerberus is planning though (seems a shame to bin the work done implementing duktape, particularly as it is so much lighter and more portable).

This is certainly possible, and is one of the unspoken goals of the JS abstraction layer.  Abstracting Duktape will be more difficult than CC/JSC because it uses a stack model instead of direct references to values, but it can be done with clever use of Duktape's global stash and heap pointers.  So we'll see where this goes. :)
Title: Re: miniSphere 4.8.4
Post by: Rhuan on August 23, 2017, 12:55:19 pm
I had in my mind the idea that ChakraCore could be selected as a compile time CC or duktape depending what you're building for. duktape gives you a smaller more portable build that uses less resources, whereas CC gives you ES6 and higher performance.

Not sure if this is what Fat Cerberus is planning though (seems a shame to bin the work done implementing duktape, particularly as it is so much lighter and more portable).

This is certainly possible, and is one of the unspoken goals of the JS abstraction layer.  Abstracting Duktape will be more difficult than CC/JSC because it uses a stack model instead of direct references to values, but it can be done with clever use of Duktape's global stash and heap pointers.  So we'll see where this goes. :)

To avoid spending forever abstracting duktape maybe just wrap most of its calls in "#ifdef"s?

EDIT: also I saw your question on the CC github, were you looking for this: https://github.com/Microsoft/ChakraCore/wiki/JsGetOwnPropertySymbols
Title: Re: miniSphere 4.8.4
Post by: Fat Cerberus on August 23, 2017, 01:06:10 pm
No, that function doesn't do what I need, that just tells me which symbol properties are set for an object, like Object.keys() but for symbols.  What I wanted was the value of the built-in Symbol.iterator so I can use it as a property key during API registration.

Worse comes to worst I can just do:
Code: [Select]
iter_symbol = js_value_new_eval("Symbol.iterator");

But that's kind of awkward (eval is evil) and I was looking for a cleaner method.
Title: Re: miniSphere 4.8.4
Post by: Rhuan on August 23, 2017, 01:27:30 pm
Sorry I don't quite follow, and to be honest I'm only guessing at answers, my experience with CC = 2-3 hours reading about it and that's it. I was just trying to be helpful.

I've done some searches through the source each JS object seems to be given an Iterator internally, JsGetProperty may allow you C level access to it, though yeah, I'm guessing.

Title: Re: miniSphere 4.8.4
Post by: Fat Cerberus on August 23, 2017, 01:31:14 pm
What I'm basically trying to do is, as part of registering the Sphere v2 API:

Code: (javascript) [Select]
DirectoryStream.prototype[Symbol.iterator] = dirStreamEnumerationCallback;

CC has JsGetPropertyIdFromSymbol() that I can use for this purpose but I need a JsValueRef for Symbol.iterator first.
Title: Re: miniSphere 4.8.4
Post by: Rhuan on August 23, 2017, 01:39:51 pm
JSGetOwnPropertySymbols and dump the results with printf or akin (following some conversions) and see what you find? Probably painful and tedious... But may work?
Title: Re: miniSphere 4.8.4
Post by: Fat Cerberus on August 23, 2017, 01:43:37 pm
To be clear, it's not really a huge problem: I can just do eval("Symbol.iterator") since it only happens during engine initialization while the JS context is still "trusted", so to speak.  I just would have preferred a more elegant solution is all. ;)

Don't worry, I have (almost) everything all figured out. :)
Title: Re: miniSphere 4.8.4
Post by: Fat Cerberus on August 23, 2017, 02:36:25 pm
I decided to rewrite the abstraction layer to use Duktape's stack model instead of direct-drive.  A stack machine is easy to implement and this will make it easier to support both engines with conditional #defines.  Plus I don't have to manually release references this way.
Title: Re: miniSphere 4.8.4
Post by: Rhuan on August 23, 2017, 03:44:01 pm
Intermittent drawing errors...

The worst kind of bug, at least I think it's a bug...

I've been opening and closing miniSphere a lot whilst testing map drawing and I've noticed something that could be quite an issue. Sometimes the drawing just doesn't work inexplicably.

As an E.g.  Since beginning to type this post I've started sphereRun 3 times, twice it came up perfectly and showed the sprite I'm using walking around albrook.

On the third attempt (with no changes to the code) it gave me a black screen - I got some log outputs I'd put in showing it had progressed through the code and pressing escape will quit (something in the dispatch update script the code sets) but nothing is drawn.

Other times it draws only some of what is intended but then continues to draw that subset, perhaps implying that the draw that failed was one of the steps where a layer was created hence that layer is blank but other layers are being drawn.

If relevant I'm using version 4.8.2 (I think - sorry too many releases recently for me to keep up building each one)

Any thoughts?
Title: Re: miniSphere 4.8.4
Post by: Fat Cerberus on August 23, 2017, 03:48:27 pm
Try again after upgrading to 4.8.4.  Seriously, not trying to be a wise guy here.  I fixed a bunch of bugs in 4.8.3 related to handling of projection and modelview matrices that could be the case of your problems here.
Title: Re: miniSphere 4.8.4
Post by: Rhuan on August 23, 2017, 03:55:42 pm
Try again after upgrading to 4.8.4.  Seriously, not trying to be a wise guy here.  I fixed a bunch of bugs in 4.8.3 related to handling of projection and modelview matrices that could be the case of your problems here.
Ok, I'll build 4.8.4. I note however that the issue definitely occurs with no projection matrix being used - but if there were issues with model view matrices, hopefully that will be it.

But my map loading code is probably particularly torturous in that it uses lots of transformations and generates multiple surfaces that it fills with content and converts to images and it changes shader several times.

The current set of results just seems like from time to time miniSphere (or possibly allegro) says: "no I'm not going to do that step, you're asking too much".

EDIT: I really should have a faster build method....
My current process for each new version is:
1. download zip of the source
2. replace source of previous version in project folder with new source
3. edit in the path hack
4. manually check for any new files or renamed files and update the build rules for them
5. build each component separately, remembering to remove the path hack for sphererun
Title: Re: miniSphere 4.8.4
Post by: Rhuan on August 23, 2017, 04:27:00 pm
I think I found a new issue in 4.8.4, or maybe it's an intentional change?

I've found I need to rewrite calls like this:
require('assert');
as
require('#/modules/assert.js');

Any attempts to call the mini_rt modules in the "normal" way just throws a not found error.

I've tested this using sphereRun (which has no pathhack) as well as miniSphere with the path hack so I'm confident it's not related to that - also #/modules/ works, which surely uses the same function under the hood?
Title: Re: miniSphere 4.8.4
Post by: Fat Cerberus on August 23, 2017, 04:30:11 pm
I reorganized the modules recently, did you start from a clean source?  It might be picking up the old layout and getting confused...
Title: Re: miniSphere 4.8.4
Post by: Rhuan on August 23, 2017, 04:34:51 pm
Yeah I hadn't twigged that you'd changed the folder structure - I'd put the newly compiled binaries into my existing sphere folder and not changed the folder layout so of course it didn't work.
Title: Re: miniSphere 4.8.4
Post by: Rhuan on August 23, 2017, 04:52:03 pm
As mentioned in my MSEngine topic, I've noticed an issue with the screen shimmering, I thought it was because of how I was playing with textures too much but it hasn't gone away when I've simplified the render process :(.

On a different topic I'm afraid the intermittent drawing failures remain, also I've been getting seg faults that may or may not be related, this first one came after a test in which nothing was displayed on the screen, I then pressed escape to quit and that triggered the crash.
Code: [Select]
Crashed Thread:        3

Exception Type:        EXC_CRASH (SIGABRT)
Exception Codes:       0x0000000000000000, 0x0000000000000000
Exception Note:        EXC_CORPSE_NOTIFY

Application Specific Information:
abort() called
*** error for object 0x7ffcb9e58720: incorrect checksum for freed object - object was probably modified after being freed.
 

Thread 0:: Dispatch queue: com.apple.main-thread
0   libsystem_kernel.dylib        0x00007fffdd02a34a mach_msg_trap + 10
1   libsystem_kernel.dylib        0x00007fffdd029797 mach_msg + 55
2   com.apple.CoreFoundation      0x00007fffc7784434 __CFRunLoopServiceMachPort + 212
3   com.apple.CoreFoundation      0x00007fffc77838c1 __CFRunLoopRun + 1361
4   com.apple.CoreFoundation      0x00007fffc7783114 CFRunLoopRunSpecific + 420
5   com.apple.HIToolbox           0x00007fffc6ce3ebc RunCurrentEventLoopInMode + 240
6   com.apple.HIToolbox           0x00007fffc6ce3cf1 ReceiveNextEventCommon + 432
7   com.apple.HIToolbox           0x00007fffc6ce3b26 _BlockUntilNextEventMatchingListInModeWithFilter + 71
8   com.apple.AppKit              0x00007fffc527ca54 _DPSNextEvent + 1120
9   com.apple.AppKit              0x00007fffc59f87ee -[NSApplication(NSEvent) _nextEventMatchingEventMask:untilDate:inMode:dequeue:] + 2796
10  com.apple.AppKit              0x00007fffc52713db -[NSApplication run] + 926
11  sphererun                     0x000000010c78913e _al_osx_run_main + 1312
12  sphererun                     0x000000010c7df5c9 main + 16
13  libdyld.dylib                 0x00007fffdcf03235 start + 1

Thread 1:
0   libsystem_kernel.dylib        0x00007fffdd03244e __workq_kernreturn + 10
1   libsystem_pthread.dylib       0x00007fffdd11c48e _pthread_wqthread + 1023
2   libsystem_pthread.dylib       0x00007fffdd11c07d start_wqthread + 13

Thread 2:
0   libsystem_kernel.dylib        0x00007fffdd03244e __workq_kernreturn + 10
1   libsystem_pthread.dylib       0x00007fffdd11c621 _pthread_wqthread + 1426
2   libsystem_pthread.dylib       0x00007fffdd11c07d start_wqthread + 13

Thread 3 Crashed:
0   libsystem_kernel.dylib        0x00007fffdd031d42 __pthread_kill + 10
1   libsystem_pthread.dylib       0x00007fffdd11f457 pthread_kill + 90
2   libsystem_c.dylib             0x00007fffdcf97420 abort + 129
3   libsystem_malloc.dylib        0x00007fffdd091f5d szone_error + 626
4   libsystem_malloc.dylib        0x00007fffdd087f0d tiny_free_list_remove_ptr + 292
5   libsystem_malloc.dylib        0x00007fffdd09c914 tiny_free_no_lock + 1484
6   libsystem_malloc.dylib        0x00007fffdd09d0d5 free_tiny + 671
7   libGPUSupportMercury.dylib    0x00007fffd5457ecd gldDestroyTexture + 39
8   libGFXShared.dylib            0x00007fffcbbd2e3a gfxDestroyPluginTexture + 60
9   GLEngine                      0x0000000111e90801 gleFreeTextureObject + 36
10  GLEngine                      0x0000000111e6031e gleUnbindDeleteHashNamesAndObjects + 179
11  GLEngine                      0x0000000111d62b1c glDeleteTextures_Exec + 671
12  sphererun                     0x000000010c7deeec ogl_destroy_bitmap + 115
13  sphererun                     0x000000010c7ca680 al_destroy_bitmap + 179
14  sphererun                     0x000000010c7b170e al_convert_bitmap + 1066
15  sphererun                     0x000000010c7b1a1f _al_convert_to_memory_bitmap + 158
16  sphererun                     0x000000010c7dad30 destroy_display + 273
17  sphererun                     0x000000010c74bd3b screen_free + 51 (screen.c:147)
18  sphererun                     0x000000010c73c5d0 shutdown_engine + 142 (main.c:485)
19  sphererun                     0x000000010c73b97d _al_mangled_main + 1085 (main.c:154)
20  sphererun                     0x000000010c788c0a +[AllegroAppDelegate app_main:] + 23
21  com.apple.Foundation          0x00007fffc91a68ad __NSThread__start__ + 1243
22  libsystem_pthread.dylib       0x00007fffdd11c93b _pthread_body + 180
23  libsystem_pthread.dylib       0x00007fffdd11c887 _pthread_start + 286
24  libsystem_pthread.dylib       0x00007fffdd11c08d thread_start + 13

I then ran it again and got a black screen but it quit nicely when I pressed escape.

The next two attempts it ran smoothly and displayed everything.

The next attempt it displayed everything but on pressing escape I got:

Code: [Select]
Crashed Thread:        2

Exception Type:        EXC_BAD_ACCESS (SIGSEGV)
Exception Codes:       KERN_INVALID_ADDRESS at 0x0000000ffffffff8
Exception Note:        EXC_CORPSE_NOTIFY

Termination Signal:    Segmentation fault: 11
Termination Reason:    Namespace SIGNAL, Code 0xb
Terminating Process:   exc handler [0]

VM Regions Near 0xffffffff8:
    MALLOC_LARGE_REUSABLE  00000001253aa000-000000012600a000 [ 12.4M] rw-/rwx SM=PRV 
-->
    STACK GUARD            0000700009e19000-0000700009e1a000 [    4K] ---/rwx SM=NUL  stack guard for thread 1

Thread 0:: Dispatch queue: com.apple.main-thread
0   libsystem_kernel.dylib        0x00007fffdd02a34a mach_msg_trap + 10
1   libsystem_kernel.dylib        0x00007fffdd029797 mach_msg + 55
2   com.apple.CoreFoundation      0x00007fffc7784434 __CFRunLoopServiceMachPort + 212
3   com.apple.CoreFoundation      0x00007fffc77838c1 __CFRunLoopRun + 1361
4   com.apple.CoreFoundation      0x00007fffc7783114 CFRunLoopRunSpecific + 420
5   com.apple.HIToolbox           0x00007fffc6ce3ebc RunCurrentEventLoopInMode + 240
6   com.apple.HIToolbox           0x00007fffc6ce3cf1 ReceiveNextEventCommon + 432
7   com.apple.HIToolbox           0x00007fffc6ce3b26 _BlockUntilNextEventMatchingListInModeWithFilter + 71
8   com.apple.AppKit              0x00007fffc527ca54 _DPSNextEvent + 1120
9   com.apple.AppKit              0x00007fffc59f87ee -[NSApplication(NSEvent) _nextEventMatchingEventMask:untilDate:inMode:dequeue:] + 2796
10  com.apple.AppKit              0x00007fffc52713db -[NSApplication run] + 926
11  sphererun                     0x0000000107d0013e _al_osx_run_main + 1312
12  sphererun                     0x0000000107d565c9 main + 16
13  libdyld.dylib                 0x00007fffdcf03235 start + 1

Thread 1:
0   libsystem_kernel.dylib        0x00007fffdd03244e __workq_kernreturn + 10
1   libsystem_pthread.dylib       0x00007fffdd11c48e _pthread_wqthread + 1023
2   libsystem_pthread.dylib       0x00007fffdd11c07d start_wqthread + 13

Thread 2 Crashed:
0   libsystem_malloc.dylib        0x00007fffdd087ecc tiny_free_list_remove_ptr + 227
1   libsystem_malloc.dylib        0x00007fffdd09c945 tiny_free_no_lock + 1533
2   libsystem_malloc.dylib        0x00007fffdd09d0d5 free_tiny + 671
3   com.apple.AMDRadeonX4000GLDriver 0x000000010dcb75e4 0x10dc1f000 + 624100
4   libGPUSupportMercury.dylib    0x00007fffd5457eba gldDestroyTexture + 20
5   libGFXShared.dylib            0x00007fffcbbd2e3a gfxDestroyPluginTexture + 60
6   GLEngine                      0x000000010d417801 gleFreeTextureObject + 36
7   GLEngine                      0x000000010d3e731e gleUnbindDeleteHashNamesAndObjects + 179
8   GLEngine                      0x000000010d2e9b1c glDeleteTextures_Exec + 671
9   sphererun                     0x0000000107d55eec ogl_destroy_bitmap + 115
10  sphererun                     0x0000000107d41680 al_destroy_bitmap + 179
11  sphererun                     0x0000000107d2870e al_convert_bitmap + 1066
12  sphererun                     0x0000000107d28a1f _al_convert_to_memory_bitmap + 158
13  sphererun                     0x0000000107d51d30 destroy_display + 273
14  sphererun                     0x0000000107cc2d3b screen_free + 51 (screen.c:147)
15  sphererun                     0x0000000107cb35d0 shutdown_engine + 142 (main.c:485)
16  sphererun                     0x0000000107cb297d _al_mangled_main + 1085 (main.c:154)
17  sphererun                     0x0000000107cffc0a +[AllegroAppDelegate app_main:] + 23
18  com.apple.Foundation          0x00007fffc91a68ad __NSThread__start__ + 1243
19  libsystem_pthread.dylib       0x00007fffdd11c93b _pthread_body + 180
20  libsystem_pthread.dylib       0x00007fffdd11c887 _pthread_start + 286
21  libsystem_pthread.dylib       0x00007fffdd11c08d thread_start + 13

This was followed by 3 more tests where everything worked correctly.
The next attempt had a graphical anomaly where everything was drawn in the top corner of the screen as if it had been scaled down massively (I had not made any change to the code though).
Title: Re: miniSphere 4.8.4
Post by: Rhuan on August 23, 2017, 05:18:50 pm
I've tired moving the canceling of the update and render scripts into a separate function:

Code: [Select]
function close_down()
{
      Dispatch.cancel(tokens[0]);
      Dispatch.cancel(tokens[1]);
}

And then using:
Code: [Select]
Dispatch.later(0, close_down);
Where I'd previously cancelled the tokens and I haven't had any more segfaults, so maybe that's the answer to that one, just make it a rule never to cancel tokens during themselves (or maybe it's fluke that I've not had any segfaults cine).

Unfortunately though the graphical anomalies continue, I've tried removing the Scalex2 but that hasn't helped.

EDIT: correction - removing the Scalex2 seems to have stopped the black screens where nothing draws and the times where one layer selectively doesn't draw. Just the shimmer left.

I think there may be an issue with using too many shader switches where sometimes allegro or open_gl just can't cope.
Title: Re: miniSphere 4.8.4
Post by: Fat Cerberus on August 23, 2017, 05:48:50 pm
Thanks for investigating, about a month ago I fixed some segfault bugs in the Dispatch handling, so it's possible errors remain.

Not sure about the graphical glitches though.
Title: Re: miniSphere 4.8.4
Post by: Fat Cerberus on August 23, 2017, 07:54:26 pm
Looking at your crash logs, it seems like it's crashing while freeing textures?  And the cause is apparently a use-after-free.  So maybe Allegro is mismanaging something...

Any chance you could send me a project I can use to try and reproduce?
Title: Re: miniSphere 4.8.4
Post by: Fat Cerberus on August 23, 2017, 11:47:03 pm
Yeah I hadn't twigged that you'd changed the folder structure - I'd put the newly compiled binaries into my existing sphere folder and not changed the folder layout so of course it didn't work.

Huh, first time I ever heard the word "twigged", had to go look it up in the dictionary.

Anyway, yeah, sorry for the disruption.  I kept releasing engines with outdated modules because I'd update some code in #/modules and forget to copy it over to #/cell_modules, or vice versa.  Now any module that can be used in either miniSphere or Cell goes in #/runtime, so there's only one copy.
Title: Re: miniSphere 4.8.4
Post by: Rhuan on August 24, 2017, 01:54:32 am
Looking at your crash logs, it seems like it's crashing while freeing textures?  And the cause is apparently a use-after-free.  So maybe Allegro is mismanaging something...

Any chance you could send me a project I can use to try and reproduce?
I posted everything I'm working on for trying out purposes here: http://forums.spheredev.org/index.php/topic,1466.msg10043.html#msg10043

Notes
- this is with the Scale2x stuff in (which I think may not be working and may be where the killer graphical anomalies occur), to remove it comment out the body of MEngine.prototype.scale
- MEngine.prototype.loadRMPMap takes far too long to run (partly due to calling the scale function)
- this is without the Dispatch change (hence this is the version that segfaults)
- I've also noticed that there's a missing raw_data.dispose() call on line 209 of MEngine (inside MEngine.prototype.loadRMPMap.

Title: Re: miniSphere 4.8.4
Post by: Fat Cerberus on August 24, 2017, 02:39:36 am
Sure, I'll look at it tomorrow after I get some sleep.  I have a theory for what's causing the "shimmering" but I'll have to see it in action to know for sure.
Title: Re: miniSphere 4.8.4
Post by: Fat Cerberus on August 24, 2017, 09:54:07 am
I don't see any MEngine.prototype.scale() function, are you sure the version you uploaded is the broken version?

Possible bug: You have this.folder set to "@".  It isn't recommended to use a SphereFS prefix by itself like that since it won't be recognized as a directory in all contexts.  You should add a slash to it ("@/") and then open the map file using:

Code: (javascript) [Select]
var raw_data = new FileStream(FS.fullPath(name, this.folder), FileOp.Read);

http://wiki.spheredev.org/index.php/API:FS.fullPath

Which has the added benefit that it always gives you the canonical normalized SphereFS path with no extraneous slashes or relative parts.  That lets you use the string returned by a FS.fullPath() as a key to track state for different maps, for example.
Title: Re: miniSphere 4.8.4
Post by: Rhuan on August 24, 2017, 12:48:08 pm
99% sure but I will double check when I get home - as said the only consistent problem is the riples, the worse anomalies and the seg faults only happen 20-30% of the time.
Title: Re: miniSphere 4.8.4
Post by: Fat Cerberus on August 24, 2017, 02:24:33 pm
One feature I think might be useful is to extend FS.fullPath() to optionally add a file extension if none is provided and the path is not prefixed.  If there's a prefix it would be assumed to already be a full path so no extension would be added.
Title: Re: miniSphere 4.8.4
Post by: Rhuan on August 24, 2017, 04:26:36 pm
I don't see any MEngine.prototype.scale() function, are you sure the version you uploaded is the broken version?
My mistake the scaling function is just called scale, not MEngine.prototype.scale. And I've checked, the version i've uploaded is the one that crashes and/or has massive graphical anomalies (i.e. whole screen black or drawing at 1/8th of the size it should) ~30% of the time. I think it may be that the scale 2x shader programme is just too much for my graphics card sometimes particularly in the context it's used in.
Title: Re: miniSphere 4.8.4
Post by: Fat Cerberus on August 24, 2017, 04:27:41 pm
Alrighty, I'll do some more testing then to see if I can find the issue.
Title: Re: miniSphere 4.8.4
Post by: Rhuan on August 24, 2017, 04:43:36 pm
The screen shimmering issue has me more worried now, the short answer to the big graphical anomalies problem seems to be: don't run an unnecessary scaling shader programme and/or maybe there's an intermittent error in said shader programme, I can do without it or I can work on tweaking it later.

Whereas the shimmering problem is there whatever I do, the only way I can find to avoid it is to not move around a map, that's a bit more fundamental, vsync helps a little but doesn't make it go away.

I feel bad that it feels like I'm endlessly asking you to help with bugs :(
Title: Re: miniSphere 4.8.4
Post by: Fat Cerberus on August 24, 2017, 04:51:28 pm
I feel bad that it feels like I'm endlessly asking you to help with bugs :(

Don't.  Seriously. :)  miniSphere was developed in the first place because I was fed up with all the glitches and crash bugs in Sphere 1.5 (towards the end, when Spectacles was pushing the engine to its limits, I saw a LOT of segfaults).  So if there's any chance there's a bug in the engine that I can fix, I'm more than happy to try to hunt it down.
Title: Re: miniSphere 4.8.4
Post by: Rhuan on August 24, 2017, 05:42:53 pm
I feel bad that it feels like I'm endlessly asking you to help with bugs :(

Don't.  Seriously. :)  miniSphere was developed in the first place because I was fed up with all the glitches and crash bugs in Sphere 1.5 (towards the end, when Spectacles was pushing the engine to its limits, I saw a LOT of segfaults).  So if there's any chance there's a bug in the engine that I can fix, I'm more than happy to try to hunt it down.
Thanks, it's really appreciated.

In other news... I thought I'd build cell which I did, I've been trying to build spectacles to test it but just keep getting a seg fault.

I can't get the fancy looking crash logs I get from miniSphere as this is a command line tool so the best output I can get is this:
Segmenation fault 11 (which apparently means attempt to access unassigned memory)

cell`___lldb_unnamed_symbol123$$cell:
    0x100008a10 <+0>:  pushq  %rbp
    0x100008a11 <+1>:  movq   %rsp, %rbp
    0x100008a14 <+4>:  movq   %rdi, -0x8(%rbp)
    0x100008a18 <+8>:  movq   -0x8(%rbp), %rdi
->  0x100008a1c <+12>: movq   0x10(%rdi), %rax <- EXC_BAD_ACCESS (code =1, address =0x10)
    0x100008a20 <+16>: popq   %rbp
    0x100008a21 <+17>: retq  
    0x100008a22 <+18>: nopw   %cs:(%rax,%rax)
Title: Re: miniSphere 4.8.4
Post by: Rhuan on August 24, 2017, 06:59:57 pm
I stuck in a stack of printfs to trace the segfault I'm getting in cell.

From main we get safely through parsing the arguments and into build_new, from there it goes to fs_new, in that function it manages root_path, game_path and app_path but cannot do system_path.

To make the system path the funtion path_rebase is called, the first few lines of this run including the malloc but then the function path_num_hops is called and the seg fault occurs before that function returns.
Title: Re: miniSphere 4.8.4
Post by: Fat Cerberus on August 24, 2017, 08:48:03 pm
path_num_hops() is very simple:
https://github.com/fatcerberus/minisphere/blob/v4.8.4/src/shared/path.c#L177-L181

The only way that should segfault is if the path pointer passed in is null.  Hmm...
Title: Re: miniSphere 4.8.4
Post by: Fat Cerberus on August 24, 2017, 08:53:30 pm
I have a suspicion path_new_self() (returns path to calling executable) is returning a null pointer.  The method to get the path on Linux probably doesn't work in macOS:

https://github.com/fatcerberus/minisphere/blob/v4.8.4/src/shared/path.c#L66-L84
Title: Re: miniSphere 4.8.4
Post by: Flying Jester on August 24, 2017, 10:33:30 pm
That's not a great method even on Linux, the proc filesystem is not very consistent or portable even between kernel versions.
Title: Re: miniSphere 4.8.4
Post by: Rhuan on August 25, 2017, 02:04:03 am
Ok, resolved that but hit another seg fault later, I'll trace it through myself and post the solution when done.
Title: Re: miniSphere 4.8.4
Post by: Fat Cerberus on August 25, 2017, 11:26:55 am
I got JSAL almost 100% functional, I can push and pop values, execute JS code (even ES6 stuff like arrow functions!), call JS functions, define native calls, etc.  Once I finish working all the kinks out I'll replace all the Duktape calls with their JSAL equivalents and see if everything blows up in my face  :P
Title: Re: miniSphere 4.8.4
Post by: Rhuan on August 25, 2017, 01:53:04 pm
I got JSAL almost 100% functional, I can push and pop values, execute JS code (even ES6 stuff like arrow functions!), call JS functions, define native calls, etc.  Once I finish working all the kinks out I'll replace all the Duktape calls with their JSAL equivalents and see if everything blows up in my face  :P

Good work.
Title: Re: miniSphere 4.8.4
Post by: Fat Cerberus on August 25, 2017, 03:26:53 pm
miniSphere is actually the easy part of the upgrade: With a different JS engine in use, SSj will need to be rewritten pretty much from the ground up since the debug interface will be different.  CC *does* have a debugging API at least, which is something I made sure to check before I started. :)
Title: Re: miniSphere 4.8.4
Post by: Rhuan on August 25, 2017, 03:58:32 pm
With cell for mac I've got it to find itself but it dies later trying to build a path for cellscript.mjs.

I can't see what input it's using but it seems to be getting to the bottom of path_rebase where it calls path_collapse but passing a null pointer to path_collapse,
Title: Re: miniSphere 4.8.4
Post by: Fat Cerberus on August 25, 2017, 04:04:36 pm
Is there any way you could build Cell with debug symbols (-g switch) so you could get a stack trace with function names?
Title: Re: miniSphere 4.8.4
Post by: Fat Cerberus on August 25, 2017, 04:31:49 pm
I managed to emulate Duktape's exception model, where JS errors can be thrown directly from C code.  Here's a test function:

Code: (JS) [Select]
static int
js_foo(int num_args, bool is_ctor)
{
const char* str;
int         num;

printf("foo() called with %d args\n", num_args);
str = jsal_require_string(1);
num = jsal_require_int(2);
printf("%s %d\n", str, num);
return 0;
}

Code: (JS) [Select]
jsal_init();
jsal_push_global_object();
jsal_push_function(js_foo, "foo", 0);
jsal_set_named_property(-2, "foo");
jsal_pop(1);
jsal_push_eval("foo('pigs!', '812');");
jsal_uninit();

Output:
Code: [Select]
foo() called with 2 args
JSAL exception thrown from unguarded C code!
-> TypeError: '812' is not a number

The second printf in foo() is not reached because jsal_require_int(2) throws an error first.  This will make it possible to just plug in JSAL calls in place of Duktape ones with very little refactoring.
Title: Re: miniSphere 4.8.4
Post by: Rhuan on August 25, 2017, 05:14:24 pm
So JSAL = JavaScript Abstraction Layer?

I'm not sure if I like the idea of a stack machine myself but good work getting it going nonetheless.

Regarding my attempts to build cell:
Debug symbols was on the whole time but seemingly not working - I assumed this was due to xcode and apple debugging not playing nice with unix-style apps (i.e. ones without a  bundle.

BUT then I realised that "strip symbols" was also on, xcode has too many defaults.

THe trace is overly large so here are the highlights in reverse order:
13: inside int main:
if (!build_eval(build, "Cellscript.mjs") && !build_eval(build, "Cellscript.js"))
12: inside bool build_eval:
duk_get_prop_string(build->js_context, -1, "fileName");
11: duk_get_prop_string
10: duk_get_prob
9: duk_hobject_getprop
8: duk_err_handle_getprop
7: duk_err_handle_error_fmt
6: duk_err_longjmp
5: duk_uncaught_error_aware
3: duk_fatal_raw
2: abort
Title: Re: miniSphere 4.8.4
Post by: Fat Cerberus on August 25, 2017, 05:34:24 pm
Yep, JavaScript Abstraction Layer.

That's interesting, it's an uncaught JS exception that's crashing it.

As for the stack machine: It wasn't my first choice, but I decided it was ultimately better to do it that way since I won't have to restructure all the JS-related calls, it can remain mostly as-is and I just have to rename the functions.  Otherwise I have to do a ton of refactoring to the entire Vanilla and Pegasus API implementations and surely create lots of bugs in the process.  Any bugs I create this way would be in JSAL itself and therefore easier to track down.
Title: Re: miniSphere 4.8.4
Post by: Fat Cerberus on August 25, 2017, 07:41:59 pm
It looks like the same bug that I just fixed in miniSphere: it's trying to get the filename and line number of an error but fails because the thing on top of the stack isn't an object.  I have no clue why that would happen though...
Title: Re: miniSphere 4.8.4
Post by: Fat Cerberus on August 26, 2017, 02:56:00 am
I keep thinking I'm done with JSAL and then realizing I need more object manipulation functions... :P

I think I'm going to try converting Cell to use JSAL (and by extension CC) first, before tackling miniSphere.  Cell has a much smaller API than Sphere, so there's a lot less surface area to test and it'll give me a good idea of how dramatic the performance boost is, as building Specs involves transpilation.

One feature of Duktape that I can't figure out how to replicate is being able to store "hidden" properties on an object which can be accessed from native code but not in JS.  I've seen a few mentions of "internal properties" in the CC documentation, but haven't found any APIs to work with them...
Title: Re: miniSphere 4.8.4
Post by: Rhuan on August 26, 2017, 05:48:58 am
One feature of Duktape that I can't figure out how to replicate is being able to store "hidden" properties on an object which can be accessed from native code but not in JS.  I've seen a few mentions of "internal properties" in the CC documentation, but haven't found any APIs to work with them...
I've had a look and can't find anything either.

The internal properties information I found seemed to me to relate to specific internal chakra core data not anything you could create via their runtime.
Title: Re: miniSphere 4.8.4
Post by: Rhuan on August 26, 2017, 02:42:48 pm
Had a thought, not for the near future but if we want to push Sphere to being more flexible/more powerful - particularly if full 3d support comes in...

Possible extra feature: Multi-threading - with CC you can create multiple runtimes which will run in multiple threads, and then you can create a couple of functions to pass information between them.

(Maybe for version 6.0....)
Title: Re: miniSphere 4.8.4
Post by: Fat Cerberus on August 26, 2017, 03:33:21 pm
After spending pretty much all day working on it, I'm so close to having Cell working with CC I can feel it. :D

There are a few weird errors being thrown that make no sense, I suspect my stack handling in JSAL is buggy.  For example `require()` fails with an error "can't set __esModule on undefined or null".  But the fact I got that far means it successfully gets through parsing the Cellscript and is trying to run it.  This is awesome!
Title: Re: miniSphere 4.8.4
Post by: Rhuan on August 26, 2017, 03:35:36 pm
After spending pretty much all day working on it, I'm so close to having Cell working with CC I can feel it. :D

There are a few weird errors being thrown that make no sense, I suspect my stack handling in JSAL is buggy.  For example `require()` fails with an error "can't set __esModule on undefined or null".  But the fact I got that far means it successfully gets through parsing the Cellscript and is trying to run it.  This is awesome!
Sounding good.

I should really get the existing cell to compile on macOS so I can build CC Cell when it's ready...
Title: Re: miniSphere 4.8.4
Post by: Fat Cerberus on August 27, 2017, 02:54:07 am
Code: [Select]
C:\src\spectacles-i>cell -rd
Cell X.X.X Sphere packaging compiler (x64)
the JavaScript-powered build engine for Sphere
(c) 2015-2017 Fat Cerberus

setting up Cellscript environment...
evaluating 'Cellscript.mjs'...
   W: no existing files match 'lib/*.js'
building targets...
   transpiling '@/scripts/battleAI/headlessHorse.js'...
   transpiling '@/scripts/battleAI/robertII-2.js'...
   transpiling '@/scripts/battleAI/robertII.js'...
   transpiling '@/scripts/battleEngine/battle.js'...
   transpiling '@/scripts/battleEngine/battleActor.js'...
   transpiling '@/scripts/battleEngine/battleAI.js'...
   transpiling '@/scripts/battleEngine/battleHUD.js'...
   transpiling '@/scripts/battleEngine/battleScreen.js'...
   transpiling '@/scripts/battleEngine/battleUnit.js'...
   transpiling '@/scripts/battleEngine/fieldCondition.js'...
   transpiling '@/scripts/battleEngine/item.js'...
   transpiling '@/scripts/battleEngine/moveMenu.js'...
   transpiling '@/scripts/battleEngine/mpPool.js'...
   transpiling '@/scripts/battleEngine/skill.js'...
   transpiling '@/scripts/battleEngine/spriteImage.js'...
   transpiling '@/scripts/battleEngine/stat.js'...
   transpiling '@/scripts/battleEngine/statusEffect.js'...
   transpiling '@/scripts/battleEngine/targetMenu.js'...
   transpiling '@/scripts/battleEngine/weapon.js'...
   transpiling '@/scripts/gameDef/animations.js'...
   transpiling '@/scripts/gameDef/battles.js'...
   transpiling '@/scripts/gameDef/characters.js'...
   transpiling '@/scripts/gameDef/conditions.js'...
   transpiling '@/scripts/gameDef/game.js'...
   transpiling '@/scripts/gameDef/items.js'...
   transpiling '@/scripts/gameDef/maps.js'...
   transpiling '@/scripts/gameDef/math.js'...
   transpiling '@/scripts/gameDef/moveEffects.js'...
   transpiling '@/scripts/gameDef/skills.js'...
   transpiling '@/scripts/gameDef/stats.js'...
   transpiling '@/scripts/gameDef/statuses.js'...
   transpiling '@/scripts/gameDef/weapons.js'...
   transpiling '@/scripts/gameOverScreen.js'...
   transpiling '@/scripts/main.js'...
   transpiling '@/scripts/maps/main.js'...
   transpiling '@/scripts/maps/Portentia.js'...
   transpiling '@/scripts/maps/Testville.js'...
   transpiling '@/scripts/menuStrip.js'...
   transpiling '@/scripts/party.js'...
   transpiling '@/scripts/scenelets.js'...
   transpiling '@/scripts/session.js'...
   transpiling '@/scripts/testCases/brucesStory.js'...
   transpiling '@/scripts/testHarness.js'...
   transpiling '@/scripts/titleScreen.js'...
   transpiling '@/scripts/battleEngine/ui.js'...
   transpiling '@/scripts/inGameClock.js'...
   installing '@/images/battleBackground.png'...
   installing '@/images/gameOverScreen.png'...
   installing '@/images/splashScreen.png'...
   installing '@/images/titleCard.png'...
   installing '@/images/titleScreen.png'...
   installing '@/maps/main.rmp'...
   installing '@/maps/Portentia.rmp'...
   installing '@/maps/Testville.rmp'...
   installing '@/maps/Portentia.rts'...
   installing '@/maps/TestvilleTiles.rts'...
   installing '@/music/basicInstinct.ogg'...
   installing '@/music/gameOver.ogg'...
   installing '@/music/manorBoss.ogg'...
   installing '@/music/nightmareBattle.ogg'...
   installing '@/music/thePromise.ogg'...
   installing '@/music/chartreuse.mp3'...
   installing '@/spritesets/battlers/Amanda.rss'...
   installing '@/spritesets/battlers/Bruce.rss'...
   installing '@/spritesets/battlers/Elysia.rss'...
   installing '@/spritesets/battlers/H. Horse.rss'...
   installing '@/spritesets/battlers/Justin.rss'...
   installing '@/spritesets/battlers/Katelyn.rss'...
   installing '@/spritesets/battlers/Lauren.rss'...
   installing '@/spritesets/battlers/Lumisquirrel.rss'...
   installing '@/spritesets/battlers/maggie.rss'...
   installing '@/spritesets/battlers/maggie_hippo.rss'...
   installing '@/spritesets/battlers/Robert.rss'...
   installing '@/spritesets/battlers/Scott T.rss'...
   installing '@/spritesets/battlers/Scott.rss'...
   installing '@/spritesets/battlers/Victor.rss'...
   installing '@/spritesets/battlers/Xemnas.rss'...
   installing '@/spritesets/invisible.rss'...
   installing '@/sounds/munch.wav'...
   installing '@/icon.png'...
cleaning up old build artifacts...
writing Sphere manifest files...
writing source map...
0 error(s), 1 warning(s).

:smile:
Title: Re: miniSphere 4.8.4
Post by: Rhuan on August 27, 2017, 04:47:18 am
Good work, now... How much slower was it than Duktape?
Title: Re: miniSphere 4.8.4
Post by: Fat Cerberus on August 27, 2017, 09:41:59 am
Good work, now... How much slower was it than Duktape?

:laughing:

I would say it was slower by a factor of about -10x. :P
Title: Re: miniSphere 4.8.4
Post by: Rhuan on August 27, 2017, 10:27:18 am
Good work, now... How much slower was it than Duktape?

:laughing:

I would say it was slower by a factor of about -10x. :P
I was hoping for more, that's the low end of the benchmarks I did, but ah well still pretty amazing, and TBH when optimised correctly most things I've been scripting don't need that level of speed - but it is nice to have it in reserve.

One question though, why Babel not TypeScript? I thought you switched to babel because typescript took too long to run in duktape couldn't you switch back now? Is Cell flexible enough to use either with a cellscript flag?
Title: Re: miniSphere 4.8.4
Post by: Fat Cerberus on August 27, 2017, 10:37:29 am
It was actually the reverse - TypeScript is faster than Babel when type checking is off--i.e. when it's just used as a vanilla ES6 transpiler.  However, Babel supports more ES6 features (see the kangax table) and detects more errors (like import/export in a .js file) that TypeScript completely ignores when type checking is off.  The problem was that Babel takes about 5x longer to process a script, so I went back and forth for a while but eventually settled on using TS in plain-JS mode.

Now Babel runs faster than TS did under Duktape. :)

Another problem with TypeScript is that for type checking to work, the compiler needs every single .ts file in the project at once--so just by changing one file you have to rebuild your whole game.  Even with a fast JS engine running it, that's not going to be very fun.
Title: Re: miniSphere 4.8.4
Post by: Rhuan on August 27, 2017, 10:47:44 am
Hmm, I see, not sure on the speed point, I guess it depends how much code there is and how frequently a retranspile is needed - is cell with Babel set up to only re-transpile the documents that have changed?

Other thoughts:
1. Per the Kangax table pure CC actually has higher ES6 compatability than either Babel or Typescript, with only one area it's behind on that being reg.expresion prototypes

2. Is the transpiling step even needed with CC?

3. For people who like the idea of working typescript can a choice of Babel or Typescript be specified in a cellscript? Without needing many other changes?

4. I note that despite point 2 I'm very keen on the idea of cell as I want to write cellscript Tools to package maps and sprites into new formats I'm designing that will be loaded a lot faster than RMP, RSS or Tiled JSON files. (I also want to see if I can make a cellscript tool to strip debug code out of my scripts for release versions)

EDIT: extra note on typescript compatibility - the Kangax table's footnote says they're looking at Typescript 1.8, version 2.4 is the latest stable release on typescriptlang.org, they support several features that Kangax says they don't support.
Title: Re: miniSphere 4.8.4
Post by: Fat Cerberus on August 27, 2017, 10:58:03 am
Hmm, I see, not sure on the speed point, I guess it depends how much code there is and how frequently a retranspile is needed - is cell with Babel set up to only re-transpile the documents that have changed?

Other thoughts:
1. Per the Kangax table pure CC actually has higher ES6 compatability than either Babel or Typescript, with only one area it's behind on that being reg.expresion prototypes

2. Is the transpiling step even needed with CC?

3. For people who like the idea of working typescript can a choice of Babel or Typescript be specified in a cellscript? Without needing many other changes?

4. I note that despite point 2 I'm very keen on the idea of cell as I want to write cellscript Tools to package maps and sprites into new formats I'm designing that will be loaded a lot faster than RMP, RSS or Tiled JSON files. (I also want to see if I can make a cellscript tool to strip debug code out of my scripts for release versions)

Yeah, only the changed files get recompiled, unless you pass the -r switch for a full rebuild.  That's true even in the 4.8.4 which uses TS, with type checking off I can compile individual scripts with no problem.  It's just the strong typing that requires passing in everything at once.

1. Makes sense, every single major engine is ahead of Babel nowadays, it's just that it was nice to be able to get basics like arrow functions and such in Duktape since it's still primarily an ES5 engine.

2. Technically no, transpilation will no longer be necessary (which is very nice!), although it might be useful if you want to compile a game that runs on earlier miniSphere versions.  That's kind of academic though since I keep making breaking changes to the API. ;)

3. It would just be a matter of coding a TypeScript tool using Cell's Tool API and using that in place of transpile().  Cell is fully programmable with JS so you can pretty much do anything you want with it.  It exposes the same FS API that miniSphere does, too.

One big benefit to this change is that runtime modules will be able to use ES6 syntax now.  You have no idea how many times I've wanted to type an arrow function while coding a runtime module and had to stop myself because otherwise the module wouldn't load. :P  Arrow functions are especially tempting when using from().
Title: Re: miniSphere 4.8.4
Post by: Rhuan on August 27, 2017, 11:29:55 am
All that aside, I'm going to stick to old ES5 JS for now, once I have a working release of the Engine code I'm writing in ES5 I'll consider what's best for the first complete refactor I choose to do.
Title: Re: miniSphere 4.8.4
Post by: Fat Cerberus on August 27, 2017, 03:24:51 pm
Started on migrating miniSphere to CC today.  Let's just say my local copy of the codebase is in shambles right now. :P

Most of the work is just straight Find and Replace but some stuff needs to be refactored.  Very tedious work, but the good part is thanks to JSAL if I decide to change engines in the future, I won't have to rewrite half the engine again.

Here's what's going to be fun: the debugger.  With Duktape I could just open a socket and wire up Duktape's debugger protocol to it with callbacks.  CC has an actual debugging API so I'm going to have to figure out how to make miniSphere and SSj communicate myself.
Title: Re: miniSphere 4.8.4
Post by: Rhuan on August 27, 2017, 06:10:20 pm
Hopefully it's not been too tedious.

I'm really looking forward to seeing this in action - I'm probably going to try and build some stupidly detailed physics sim to see if I can get a slow down.
Title: Re: miniSphere 4.8.4
Post by: Fat Cerberus on August 27, 2017, 08:28:03 pm
I'm debating whether to release the CC miniSphere as 4.9 or 5.0.  It's a pretty big change that definitely warrants a new major version, it's just that I've promised an API freeze for mS 5.0 and I'm not quite ready to set everything in stone yet.
Title: Re: miniSphere 4.8.4
Post by: Rhuan on August 27, 2017, 08:38:04 pm
I'm debating whether to release the CC miniSphere as 4.9 or 5.0.  It's a pretty big change that definitely warrants a new major version, it's just that I've promised an API freeze for mS 5.0 and I'm not quite ready to set everything in stone yet.
Which parts of the API are still in flux?
Title: Re: miniSphere 4.8.4
Post by: Fat Cerberus on August 27, 2017, 09:38:08 pm
In particular, the shader API needs work.
https://github.com/fatcerberus/minisphere/issues/192

Not sure what else, I'll have to skim through sphere2-core-api.txt when I get home to jog my memory.
Title: Re: miniSphere 4.8.4
Post by: Fat Cerberus on August 28, 2017, 03:23:51 am
Almost got miniSphere going, it runs games but there are a lot of glitches and in a fully optimized Release build, it actually crashes outright.  Not sure what's wrong yet and I don't feel like pulling an all-nighter to figure it out.  I'll revisit it later.
Title: Re: miniSphere 4.8.4
Post by: Rhuan on August 28, 2017, 09:40:16 am
I'm debating whether to release the CC miniSphere as 4.9 or 5.0.  It's a pretty big change that definitely warrants a new major version, it's just that I've promised an API freeze for mS 5.0 and I'm not quite ready to set everything in stone yet.
Which parts of the API are still in flux?
Having looked over the issues list I think that the CC version should be 4.9 if you're going to release it soon, save 5.0 until various other aspects are done - i.e. most of the issues log, unless you want to push all of that to 6.0 - just means that the version number will tick up very quickly.

In particular, the shader API needs work.
https://github.com/fatcerberus/minisphere/issues/192

Not sure what else, I'll have to skim through sphere2-core-api.txt when I get home to jog my memory.
Not sure what I think of a shader api re-work. At their core shaders are not n00bie friendly and won't be unless there is a very hand-holdy tutorial written, the key reason for this is they require the use of a second programming language with different syntax and a few other importantly different idiosyncrasies.

Being able to distinguish floats vs ints is very important for shaders, so I actually think having Duck typing with uniform setting would be harder to work with.

One option I suppose would be a setUniform method that takes a type as the first parameter, then at least the type is clearly written in the JS file.

Almost got miniSphere going, it runs games but there are a lot of glitches and in a fully optimized Release build, it actually crashes outright.  Not sure what's wrong yet and I don't feel like pulling an all-nighter to figure it out.  I'll revisit it later.
Sounds like great progress though, so good work.
Title: Re: miniSphere 4.8.4
Post by: Fat Cerberus on August 28, 2017, 01:26:26 pm
I found the bug, I was mismanaging JsRefs.  Calling pop_value() releases the reference, and then I was just re-inserting it into the stack without calling JsAddRef() and the garbage collector got it.  Hence the crashes.

Not sure why I couldn't get it to crash in a debug build though...
Title: Re: miniSphere 4.8.4
Post by: Fat Cerberus on August 29, 2017, 12:22:57 am
The engine's up and running finally.  I verified by converting the Prim module to an actual class with static methods, worked like a charm. :D

Now I just have to fix the crazy memory leaks.  My finalizers aren't getting called because I didn't figure out how to make them work yet and so everything gets leaked.  Depending on the game you can sometimes hit 1GB of RAM usage--and climbing--within seconds.
Title: Re: miniSphere 4.8.4
Post by: Fat Cerberus on August 29, 2017, 02:37:58 am
Got the finalizers to run.  I can't get them to run on shutdown but it's not a big deal since the OS will clean everything up at that point anyway.  Both Sphere v1 and v2 games run flawlessly now.

The engine definitely seems a lot snappier than it was with Duktape, that's for sure.

There are two things left in the engine that still don't work:
Title: Re: miniSphere 4.8.4
Post by: Eggbertx on August 29, 2017, 11:09:59 am
I had a weird dream about miniSphere, CC, and SM last night...
Title: Re: miniSphere 4.8.4
Post by: Fat Cerberus on August 29, 2017, 11:22:36 am
...now I want to hear about it. ;D

In other news, CC is *fast*.  Remember the SoundStream demo I posted here?
http://forums.spheredev.org/index.php/topic,1400.msg9870.html#msg9870

Under Duktape that spiked my CPU usage to around 15% (almost completely saturating a core), and didn't complete buffering the mp3 file until the song was nearly over.  Under CC, total CPU utilization is around 3% and the stream is completely buffered by about 30 seconds in.  If I remove the delay, decoding the whole file takes no more than 5 seconds.
Title: Re: miniSphere 4.8.4
Post by: Eggbertx on August 29, 2017, 02:06:42 pm
It's kind of difficult to describe, and it was very abstract. They were more than just software, but not really physical. They just kind of...existed.

Also, that's pretty awesome! Do you think it wold be possible to optimize it even more to shorten the amount of time, or would that require native decoding?
Title: Re: miniSphere 4.8.4
Post by: Fat Cerberus on August 29, 2017, 03:55:29 pm
I'm at the mercy of how fast the decoder can send me data.  I'm just taking the data it sends me in callbacks and stuffing them into the stream with SoundStream#write().  There's not much more I can do to optimize that on my end.

Aurora.js does seem to be inefficient, though, and for whatever reason the callbacks aren't async except for the first one, so it blocks execution until the file finishes buffering.  I'd probably be better off using a native decoder.
Title: Re: miniSphere 4.8.4
Post by: Fat Cerberus on August 29, 2017, 05:35:56 pm
@Rhuan Since you already looked through the GitHub issues list, is there anything in there that you would like to be prioritized for implementation in mS 4.9?
Title: Re: miniSphere 4.8.4
Post by: Rhuan on August 29, 2017, 05:40:08 pm
@Rhuan Since you already looked through the GitHub issues list, is there anything in there that you would like to be prioritized for implementation in mS 4.9?
Not really, based on your posts above the things I'd want in 4.9 would be miniSphere running smoothly with chakra including module loading and line numbers for script errors... and SSJ working. My view is that everything else can wait.
Title: Re: miniSphere 4.8.4
Post by: Fat Cerberus on August 29, 2017, 09:58:32 pm
I removed the core-js shim (polyfill for ES6 built-in functions, obviously no longer necessary) and the performance improved even further.  Time to build Spectacles using Babel: 2 seconds. :)

edit: Good news!  I got modules working!  miniSphere 4.9 will thus have nearly full support for ES6 without any transpilation or polyfills, AND be way faster to boot.  Now THAT'S what I call an upgrade. :D
Title: Re: miniSphere 4.8.4
Post by: Eggbertx on August 30, 2017, 02:16:04 am
I'd probably be better off using a native decoder.
Yeah, that's what I had in mind. It would obviously take some work, but I'm sure it would be worth it.
Also, I have some good news. It turns out that the issues I was having in Arch Linux with tracker files not playing in miniSphere was because of an out of date Dumb package, so that's hopefully going to come out of the testing repo (which it was recently put into) and into the main repo.
Title: Re: miniSphere 4.8.4
Post by: Fat Cerberus on August 30, 2017, 09:14:40 am
Slight complication: CC doesn't seem to provide access to the module export table through its API, so I can't make require() work with ES6 modules like it did with Babel and TS. :(
Title: Re: miniSphere 4.8.4
Post by: Eggbertx on August 30, 2017, 11:52:08 am
Would it be unreasonably difficult to implement it yourself?
Title: Re: miniSphere 4.8.4
Post by: Fat Cerberus on August 30, 2017, 12:01:10 pm
The problem is that there's seemingly no access to the module's exports from the native side--so there's nothing to latch onto.

I managed to replicate mS<=4.8 behavior of instantiating a class exported from the main module by evaluating a fake root module that imports the real one, but I don't think I can adapt that to make require() work, since I have to return the exports to the caller somehow.
Title: Re: miniSphere 4.8.4
Post by: Fat Cerberus on August 30, 2017, 12:13:45 pm
Would it be unreasonably difficult ti implement it yourself?

Hmm... come to think of it, there may be a way to make this work after all.  I could evaluate the following fake module:
Code: (javascript) [Select]
import * as RealModule from '<filename>';
global.___exports = RealModule;

And then just return the value of ___exports from require().

edit: It worked!
Title: Re: miniSphere 4.8.4
Post by: Fat Cerberus on August 30, 2017, 12:47:47 pm
Progress report!

Here's what works so far:

Here's what doesn't:

Getting require() to be able to load modules was important, as without that ability, transpiled code doesn't work, and backward compatibility is lost.  Both Babel and TypeScript transpile import statements to require() calls.
Title: Re: miniSphere 4.8.4
Post by: Fat Cerberus on August 30, 2017, 03:48:21 pm
I just want to say that being able to run this:
https://github.com/fatcerberus/kh2bar

...as-is, with no transpilers or shims whatsoever, is an awesome feeling. ;D
Title: Re: miniSphere 4.8.4
Post by: Rhuan on August 30, 2017, 04:10:32 pm
Progress report!

Here's what works so far:
  • All Sphere v1 and v2 APIs
  • Cell
  • ES6 syntax, no transpilation required
  • ES6 built-ins, no polyfill required
  • ES6 modules!
  • ES2017 async functions (!!)
  • require() can load ES modules
  • STILL backward compatible with Sphere 1.x games :D

Here's what doesn't:
  • File and line information on the error screen.  SpheRun does show a stack trace after a crash, however.
  • SSj and SSj Blue
  • Linux support; it crashes under VirtualBox at least

Getting require() to be able to load modules was important, as without that ability, transpiled code doesn't work, and backward compatibility is lost.  Both Babel and TypeScript transpile import statements to require() calls.
Good work, is it worth me checking out the latest version of the chakra branch and trying to get it going on Mac?

Also I love how you almost seem to have taken over the issues page on CC GitHub repository.
Title: Re: miniSphere 4.8.4
Post by: Fat Cerberus on August 30, 2017, 04:31:15 pm
Yeah, it should be fairly stable at this point.
Title: Re: miniSphere 4.8.4
Post by: Rhuan on August 30, 2017, 05:41:54 pm
I can't understand the benefit of an ES17 async functions - possibly as I've never written web javascript.

I similarly don't understand the benefit of promises, unless they're to do with working with working in a browser event loop?
Title: Re: miniSphere 4.8.4
Post by: Rhuan on August 30, 2017, 06:46:02 pm
Unrelated to previous post: how does new Texture(width, height, pixels); handle alpha values?
This is used in my rmp loading code to load the tileset, and there are some tiles that are meant to be partially transparent that just don't come out right.

Another unrelated point: the linux failure, could it be related to ICU? the library that CC uses for handling unicode - bizarrely when I first built CC statically it still expected to link to a dynamic copy of ICU (which is not installed by default on macos and I imagine isn't in various linux builds) building CC with command --embed-icu makes the build script download icu and link to it statically. (I couldn't see an obvious way to link statically other than using that without editing a make file myself which with CC scared me slightly)
Title: Re: miniSphere 4.8.4
Post by: Fat Cerberus on August 30, 2017, 07:08:21 pm
The Texture constructor, when passing it a buffer, should just lock the texture and copy over the pixel data (RGBA) wholesale, overwriting whatever is there.  What kind of issue are you having?  Admittedly that facility doesn't get tested very often, so there could be a bug.

Note that the pixel data should be tightly packed RGBA, i.e. stride = width.  At least I think it's RGBA...

And it might be ICU, I'll have to look into it.  The .so binary I put in dep/ is the official x64 Linux binary from the CC GitHub page.  It would be nice if we could static-link CC even for Linux, since there are no official packages yet that I could find.
Title: Re: miniSphere 4.8.4
Post by: Rhuan on August 30, 2017, 07:14:31 pm
OK so I got the CC branch to build on macOs but a few problems and it doesn't run :(
Issues to make it build:
1. Your functions asprintf and vasprintf conflict with the names of some functions in the macos version of stdio.h which gets included somewhere.
2. I had to specifically link icu as well as chakra
3. I also had to link libc++ (though this is installed as standard on macos so ok to use the system dylib)

Run time crash (on loading sphere by double clicking the app, i.e. it should in theory be trying to launch the start up game):
Code: [Select]
Thread 12 Crashed:
0   libsystem_c.dylib             0x00007fffb5df5b52 strlen + 18
1   spheredev.miniSphere          0x0000000106c6aaf7 jsal_push_new_error_va + 16 (jsal.c:1000)
2   spheredev.miniSphere          0x0000000106c5b048 jsal_error_blame + 160 (api.c:308)
3   spheredev.miniSphere          0x0000000106c8a39d js_RequireScript + 77 (vanilla.c:5054)
4   spheredev.miniSphere          0x0000000106c6bbcc do_native_call + 447 (jsal.c:1820)
5   spheredev.miniSphere          0x00000001073d01c7 Js::JavascriptExternalFunction::StdCallExternalFunctionThunk(Js::RecyclableObject*, Js::CallInfo, ...) + 199
6   spheredev.miniSphere          0x00000001075257ae amd64_CallFunction + 78
7   spheredev.miniSphere          0x00000001072eb130 void Js::InterpreterStackFrame::OP_CallCommon<Js::OpLayoutT_CallIFlags<Js::LayoutSizePolicy<(Js::LayoutSize)0> > >(Js::OpLayoutT_CallIFlags<Js::LayoutSizePolicy<(Js::LayoutSize)0> > const*, Js::RecyclableObject*, unsigned int, Js::AuxArray<unsigned int> const*) + 304
8   spheredev.miniSphere          0x000000010728e58d Js::InterpreterStackFrame::ProcessUnprofiled() + 13565
9   spheredev.miniSphere          0x000000010727f28f Js::InterpreterStackFrame::Process() + 463
10  spheredev.miniSphere          0x000000010727e912 Js::InterpreterStackFrame::InterpreterHelper(Js::ScriptFunction*, Js::ArgumentReader, void*, void*, Js::InterpreterStackFrame::AsmJsReturnStruct*) + 1138
11  spheredev.miniSphere          0x000000010727e48d Js::InterpreterStackFrame::InterpreterThunk(Js::JavascriptCallStackLayout*) + 93
12  ???                           0x000000010f740132 0 + 4554228018
13  spheredev.miniSphere          0x00000001075257ae amd64_CallFunction + 78
14  spheredev.miniSphere          0x00000001073d2c1b Js::JavascriptFunction::CallRootFunction(Js::Arguments, Js::ScriptContext*, bool) + 27
15  spheredev.miniSphere          0x0000000106eb674f JsCallFunction + 735
16  spheredev.miniSphere          0x0000000106c6a4ca jsal_call_method + 166 (jsal.c:199)
17  spheredev.miniSphere          0x0000000106c6a39a jsal_call + 53 (jsal.c:178)
18  spheredev.miniSphere          0x0000000106c6cb9e jsal_try_call + 74 (jsal.c:1525)
19  spheredev.miniSphere          0x0000000106c7c764 script_eval + 215 (script.c:99)
20  spheredev.miniSphere          0x0000000106c6d82a _al_mangled_main + 2505 (main.c:274)
21  spheredev.miniSphere          0x0000000106de4326 +[AllegroAppDelegate app_main:] + 23
22  com.apple.Foundation          0x00007fffa20628ad __NSThread__start__ + 1243
23  libsystem_pthread.dylib       0x00007fffb5fd893b _pthread_body + 180
24  libsystem_pthread.dylib       0x00007fffb5fd8887 _pthread_start + 286
25  libsystem_pthread.dylib       0x00007fffb5fd808d thread_start + 13

I may investigate further tomorrow, just posting it for now in case there's anything that jumps out at you.


Edit, point on Pixel data from previous post
Issue I'm getting is certain tiles that should be partially transparent being 100% transparent. (Image data I'm loading is pulled from a sphere v1 tileset inside an rmp)

Edit... one more point
With Chakra included this Sphere is no longer mini, the app I've just built is 42mb. (I note it's probably possible to optimise it down a bit for size but I don't know how much and I wonder if the CC enabled version may end up needing a different name...)
Title: Re: miniSphere 4.8.4
Post by: Eggbertx on August 30, 2017, 10:46:09 pm
The problem is that there's seemingly no access to the module's exports from the native side--so there's nothing to latch onto.
There's no way to do it natively via some kind of shim? This seems like it would be a definite deal breaker, unfortunately.

With Chakra included this Sphere is no longer mini, the app I've just built is 42mb. (I note it's probably possible to optimise it down a bit for size but I don't know how much and I wonder if the CC enabled version may end up needing a different name...)
How about bigMiniSphere? :P
Title: Re: miniSphere 4.8.4
Post by: Fat Cerberus on August 30, 2017, 10:49:00 pm
There's no way to do it natively via some kind of shim? This seems like it would be a definite deal breaker

That's exactly what I did end up doing, did you see my second reply?

edit: If you missed it, what I did was to have miniSphere evaluate a fake top-level module:
Code: (javascript) [Select]
import * as Module from "<module filename>"
global.___exports = Module;

...then return the value of ___exports from require().  The same technique is used to load the main class from the startup module.
Title: Re: miniSphere 4.8.4
Post by: Eggbertx on August 31, 2017, 12:22:34 am
Ah, sorry, I didn't notice that.

Also, I built minisphere in Debian, and I'm getting a segmentation fault when I exit, and I get this message in gdb (after rebuilding with debugging symbols)
Code: [Select]
4205	malloc.c: No such file or directory.
(gdb) backtrace
#0  malloc_consolidate (av=av@entry=0x7ffff64a7b00 <main_arena>) at malloc.c:4205
#1  0x00007ffff6186dca in _int_malloc (av=av@entry=0x7ffff64a7b00 <main_arena>, bytes=bytes@entry=4096) at malloc.c:3488
#2  0x00007ffff6188f34 in __GI___libc_malloc (bytes=4096) at malloc.c:2928
#3  0x00007ffff6c9d5d1 in ?? () from /usr/lib/x86_64-linux-gnu/liballegro.so.5.2
#4  0x00007ffff6c9e212 in al_create_bitmap () from /usr/lib/x86_64-linux-gnu/liballegro.so.5.2
#5  0x00007ffff6c9eb35 in al_clone_bitmap () from /usr/lib/x86_64-linux-gnu/liballegro.so.5.2
#6  0x00007ffff6ca2cc8 in al_convert_bitmap () from /usr/lib/x86_64-linux-gnu/liballegro.so.5.2
#7  0x00007ffff6ca321b in _al_convert_to_memory_bitmap () from /usr/lib/x86_64-linux-gnu/liballegro.so.5.2
#8  0x00007ffff6d08c80 in ?? () from /usr/lib/x86_64-linux-gnu/liballegro.so.5.2
#9  0x00005555555d969e in screen_free (it=0x555555913ea0) at src/minisphere/screen.c:146
#10 0x0000555555568327 in shutdown_engine () at src/minisphere/main.c:484
#11 0x000055555555e4c8 in main (argc=<optimized out>, argv=<optimized out>) at src/minisphere/main.c:167

If I try to change the resolution of the startup game, I get this as well
Code: [Select]
*** Error in `/usr/bin/minisphere': corrupted size vs. prev_size: 0x00005555564aec5f ***
Title: Re: miniSphere 4.8.4
Post by: Fat Cerberus on August 31, 2017, 01:36:00 am
This is a 4.8.4 build or...?

Also, I think I can use a similar module shim to make things work the other way around too, i.e. importing CommonJS modules.  Instead of giving the source code of the CJS module directly to the module loader (which would cause a syntax error), I instead replace it with this shim:
Code: (JavaScript) [Select]
export default const Module = require("<filename>");
Title: Re: miniSphere 4.8.4
Post by: Fat Cerberus on August 31, 2017, 10:00:41 am
I can't understand the benefit of an ES17 async functions - possibly as I've never written web javascript.

I similarly don't understand the benefit of promises, unless they're to do with working with working in a browser event loop?

They're not too useful in Sphere--yet.  They can be very useful if you have APIs that do stuff in the background, though, for example downloading files.  Suppose you have an operation that needs to download a file but can't do anything until the file is complete, however you want your game to continue running as normal.  You could implement that as an async function:

Code: (javascript) [Select]
async function downloadHugeFile(url)
{
    SSj.log(`starting to download ${url}...`);
    await downloadFile(url);  // yields to event loop
    SSj.log("download complete!");
}

The hypothetical downloadFile() function would return a promise that it resolves when the download is complete, after which point execution continues right after the await.

Notes:

Sphere has an event loop as well; in Sphere v1 you invoke it manually by calling FlipScreen().  Sphere v2 has a real, Node-like event loop that runs as long as there's something in the Dispatch queue.
Title: Re: miniSphere 4.8.4
Post by: Rhuan on August 31, 2017, 12:56:06 pm
I can't understand the benefit of an ES17 async functions - possibly as I've never written web javascript.

I similarly don't understand the benefit of promises, unless they're to do with working with working in a browser event loop?

They're not too useful in Sphere--yet.  They can be very useful if you have APIs that do stuff in the background, though, for example downloading files.  Suppose you have an operation that needs to download a file but can't do anything until the file is complete, however you want your game to continue running as normal.  You could implement that as an async function:

Code: (javascript) [Select]
async function downloadHugeFile(url)
{
    SSj.log(`starting to download ${url}...`);
    await downloadFile(url);  // yields to event loop
    SSj.log("download complete!");
}

The hypothetical downloadFile() function would return a promise that it resolves when the download is complete, after which point execution continues right after the await.

Notes:
  • async () => 812 is shorthand for: () => Promise.resolve(812);
  • await x() is shorthand for: x().then(retval => { /*everything after the await*/ })

Sphere has an event loop as well; in Sphere v1 you invoke it manually by calling FlipScreen().  Sphere v2 has a real, Node-like event loop that runs as long as there's something in the Dispatch queue.
If I understand this correctly, the downloadFile function would be run in a seperate thread and the second SSj.log call would take place in the event loop when the second thread reported that the file was downloaded?

In other words this allows you to make non-blocking api calls and have specific functions pause until the api call is complete then resume BUT other JS execution continues during the pause?

I can't really think of anything in the Sphere v1 or v2 apis that would benefit from being called in that way, if I could call another JS function with await in front of it rather than an api funciton that could be amazing but based on my reading of it it doesn't work that way.
Title: Re: miniSphere 4.8.4
Post by: Rhuan on August 31, 2017, 01:25:22 pm
I've tried running CC miniSphere with verbosity on to see all of the logs and find the crash:

I got the following:

Code: [Select]
initializing Allegro 5.2.2.0
initializing Dyad 0.2.0
initializing JavaScript
initializing dispatch manager
initializing Galileo subsystem
initializing audio subsystem
initializing input subsystem
opening kevfile #0 `#/miniSphere.conf`
    `#/miniSphere.conf` doesn't exist
reading key `keymap_Player1_MENU` from kevfile #0
SNIP - removed numerous lines about loading keys
reading key `keymap_Player4_Y` from kevfile #0
disposing kevfile #0 no longer in use
initializing sockets subsystem
    Dyad.c 0.2.0
initializing spriteset manager
initializing map engine subsystem
creating new mixer #1 at 44 kHz
    format: 2ch 44100 Hz, 16-bit
initializing JS script manager
initializing legacy support module
opening kevfile #1 `#/system.ini`
reading key `Font` from kevfile #1
loading font #0 as `#/system.rfn`
creating image #0 at 128x192
creating image #1 as 8x12 subimage of image #0
SNIP - removed numerous lines about creating subimages of image #0
creating image #256 as 8x12 subimage of image #0
reading key `WindowStyle` from kevfile #1
reading 16x16 image #257 from open file
reading 16x16 image #258 from open file
reading 16x16 image #259 from open file
reading 16x16 image #260 from open file
reading 16x16 image #261 from open file
reading 16x16 image #262 from open file
reading 16x16 image #263 from open file
reading 16x16 image #264 from open file
reading 16x16 image #265 from open file
reading key `Arrow` from kevfile #1
loading image #266 as `#/pointer.png`
reading key `UpArrow` from kevfile #1
loading image #267 as `#/up_arrow.png`
reading key `DownArrow` from kevfile #1
loading image #268 as `#/down_arrow.png`
disposing kevfile #1 no longer in use
setting up jump points for longjmp
searching for a game to launch
opening `/Xdevelopment/startup/game.sgm` from game #1
opening package #1 `/Xdevelopment/startup/game.sgm`
failed to open package #1
opening kevfile #2 `@/game.sgm`
parsing SGM manifest for game #1
reading key `name` from kevfile #2
reading key `author` from kevfile #2
reading key `description` from kevfile #2
reading key `screen_width` from kevfile #2
reading key `screen_height` from kevfile #2
reading key `script` from kevfile #2
disposing kevfile #2 no longer in use
         title: Startup
        author: tunginobi
    resolution: 320x240
       save ID: (null)
loading image #269 as `@/icon.png`
    failed to load image #269
loading image #270 as `#/icon.png`
initializing render context at 320x240
creating image #271 at 320x240
cloning image #272 from source image #270
2017-08-31 18:17:17.628 minisphere[22798:1702920] void * _Nullable NSMapGet(NSMapTable * _Nonnull, const void * _Nullable): map table argument is NULL
activating legacy shaders
opening kevfile #3 `@/keymap.conf`
    `@/keymap.conf` doesn't exist
reading key `keymap_Player1_MENU` from kevfile #3
SNIP - removed numerous lines about reading keys
reading key `keymap_Player4_Y` from kevfile #3
disposing kevfile #3 no longer in use
initializing Sphere v1 API (v2.0)
creating new mixer #2 at 44 kHz
    format: 2ch 44100 Hz, 16-bit
initializing Sphere v2 L0 API
creating new mixer #3 at 44 kHz
    format: 2ch 44100 Hz, 16-bit
loading system default font
GAME CRASH: uncaught JavaScript exception.
)
Note the game it's trying to load is just the startup game that comes with sphere, I have not edited it.

Interestingly having turned on verbosity it's stopped segfaulting, which seems utterly bizarre.
Title: Re: miniSphere 4.8.4
Post by: Fat Cerberus on August 31, 2017, 02:04:04 pm
If I understand this correctly, the downloadFile function would be run in a seperate thread and the second SSj.log call would take place in the event loop when the second thread reported that the file was downloaded?

In other words this allows you to make non-blocking api calls and have specific functions pause until the api call is complete then resume BUT other JS execution continues during the pause?

I can't really think of anything in the Sphere v1 or v2 apis that would benefit from being called in that way, if I could call another JS function with await in front of it rather than an api funciton that could be amazing but based on my reading of it it doesn't work that way.

...I think I'm going to bring the Pact class back.

You can await on anything that returns a promise.  In the case of a native API call, yeah, the download would probably run in a separate thread.  But it doesn't have to.  For example, you could do it in JavaScript using a socket.  Here's a sloppy pseudocode implementation:

Code: (javascript) [Select]
import { Pact } from 'sphere-runtime';

var pact  = new Pact();
var downloads = [];

function downloadFile()
{
    let socket = new Socket(url, 80);
    let promise = pact.promise();
    downloads.push({ socket, promise });
    return promise;
}

Dispatch.onUpdate(() => {
    for (let dl of downloads) {
        allData += dl.socket.read(dl.socket.bytesPending);
        if (downloadIsFinished) {  // whatever criteria for when a download is done
            pact.resolve(dl.promise, allData);
        }
    }
});

Then you could just do:
Code: (javascript) [Select]
async function loadInBackground()
{
    SSj.log("loading...");
    let downloadedData = await downloadFile("whatever");
    SSj.log("finished loading!");
}

A promise is basically just "I'll call you back later when I'm in a better position to give you an answer".  You don't need real threads to do that, although they do help. :)
Title: Re: miniSphere 4.8.4
Post by: Rhuan on August 31, 2017, 02:09:02 pm
Unless we're dealing with download delays or actually have multiple-threads I don't see much benefit as we can't do two things at once and generally need everything to happen sequentially anyway.

I'm afraid I don't know where to begin in debugging the crashes I've posted about above, any ideas?
Title: Re: miniSphere 4.8.4
Post by: Fat Cerberus on August 31, 2017, 02:21:39 pm
Unless we're dealing with download delays or actually have multiple-threads I don't see much benefit as we can't do two things at once and generally need everything to happen sequentially anyway.

Not necessarily.  Dispatch.now() and its siblings exist for a reason.  await is basically that, except an entire function is returning early ("yielding"), and the thing being dispatched is "continue running the paused function".  Promises are the mechanism JS uses under the hood to achieve that.

Quote
I'm afraid I don't know where to begin in debugging the crashes I've posted about above, any ideas?

Hard to say,  Going by your crash log above, it seems that the crash was ultimately in strlen, which is suspicious and indicates that either:

Is there any chance you could run the engine under Valgrind?  It will be excruciatingly slow, but if there is any memory corruption Valgrind will tell you exactly where it happens, complete with stack traces.  Make sure to compile the engine with debug symbols.
Title: Re: miniSphere 4.8.4
Post by: Rhuan on August 31, 2017, 03:20:13 pm
I tried Valgrind but it didn't tell me anything useful.

I've tried sticking printfs everywhere, it dies in the function script_eval within script.c, it reliably will run a printf on the line before:
if (!jsal_try_call(0))

But oddly it will not run a printf placed in the top of jsal_try_call - it appears to crash as that function is called, if I comment out that function call it still crashes at about the same time, it seems to me that there's a second thread that's crashing - could it be that CC is receiving the script in the wrong format and it's compiler is multi-threaded and one of those threads is crashing?

I recall the CC documentation talking about different unicode handling functions for cross platform vs windows could this be the issue?
Title: Re: miniSphere 4.8.4
Post by: Fat Cerberus on August 31, 2017, 03:35:33 pm
This is what I get out of valgrind when trying to run Cell on Ubuntu:
Code: [Select]
==852== Memcheck, a memory error detector
==852== Copyright (C) 2002-2015, and GNU GPL'd, by Julian Seward et al.
==852== Using Valgrind-3.11.0 and LibVEX; rerun with -h for copyright info
==852== Command: cell
==852==
Out of Memory
==852==
==852== Process terminating with default action of signal 6 (SIGABRT)
==852==    at 0x5E80428: raise (raise.c:54)
==852==    by 0x5E82029: abort (abort.c:89)
==852==    by 0x51F2525: Memory::X64WriteBarrierCardTableManager::Initialize() (in /usr/lib/libChakraCore.so)
==852==    by 0x4F7B63A: _GLOBAL__sub_I_RecyclerWriteBarrierManager.cpp (in /usr/lib/libChakraCore.so)
==852==    by 0x40106B9: call_init.part.0 (dl-init.c:72)
==852==    by 0x40107CA: call_init (dl-init.c:30)
==852==    by 0x40107CA: _dl_init (dl-init.c:120)
==852==    by 0x4000C69: ??? (in /lib/x86_64-linux-gnu/ld-2.23.so)
==852==
==852== HEAP SUMMARY:
==852==     in use at exit: 110,188 bytes in 24 blocks
==852==   total heap usage: 26 allocs, 2 frees, 110,723 bytes allocated
==852==
==852== LEAK SUMMARY:
==852==    definitely lost: 0 bytes in 0 blocks
==852==    indirectly lost: 0 bytes in 0 blocks
==852==      possibly lost: 304 bytes in 1 blocks
==852==    still reachable: 109,884 bytes in 23 blocks
==852==         suppressed: 0 bytes in 0 blocks
==852== Rerun with --leak-check=full to see details of leaked memory
==852==
==852== For counts of detected and suppressed errors, rerun with: -v
==852== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)
Killed

Out of Memory, before the process even gets going.  No idea what's up with that, as I have 4GB of RAM allocated to the virtual machine.  If I run Cell manually, I get this:
Code: [Select]
fatcerberus@pigcult-vm:~/src/spectacles-i$ cell
Cell X.X.X Sphere packaging compiler (x64)
the JavaScript-powered build engine for Sphere
(c) 2015-2017 Fat Cerberus

setting up Cellscript environment...
evaluating '$/Cellscript.mjs'...
   E:
SCRIPT CRASH: uncaught JavaScript exception.
   at 1:0
1 error(s), 0 warning(s).

miniSphere on the other hand segfaults with a floating point exception in the same environment (and also reports out of memory under valgrind).

You're sure Valgrind didn't report any corruption/buffer overruns?
Title: Re: miniSphere 4.8.4
Post by: Rhuan on August 31, 2017, 03:59:39 pm
It told me stuff just didn't seem useful, and didn't give me proper symbols whatever compiler options I changed.

Code: [Select]
==42230== Memcheck, a memory error detector
==42230== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
==42230== Using Valgrind-3.13.0 and LibVEX; rerun with -h for copyright info
==42230== Command: ./ccminisphere.app/contents/macos/minisphere
==42230==
--42230-- run: /usr/bin/dsymutil "./ccminisphere.app/contents/macos/minisphere"
==42230== Syscall param msg->desc.port.name points to uninitialised byte(s)
==42230==    at 0x106B8A34A: mach_msg_trap (in /usr/lib/system/libsystem_kernel.dylib)
==42230==    by 0x106B89796: mach_msg (in /usr/lib/system/libsystem_kernel.dylib)
==42230==    by 0x106B83485: task_set_special_port (in /usr/lib/system/libsystem_kernel.dylib)
==42230==    by 0x106D1F10E: _os_trace_create_debug_control_port (in /usr/lib/system/libsystem_trace.dylib)
==42230==    by 0x106D1F458: _libtrace_init (in /usr/lib/system/libsystem_trace.dylib)
==42230==    by 0x1031C69DF: libSystem_initializer (in /usr/lib/libSystem.B.dylib)
==42230==    by 0x103082A1A: ImageLoaderMachO::doModInitFunctions(ImageLoader::LinkContext const&) (in /usr/lib/dyld)
==42230==    by 0x103082C1D: ImageLoaderMachO::doInitialization(ImageLoader::LinkContext const&) (in /usr/lib/dyld)
==42230==    by 0x10307E4A9: ImageLoader::recursiveInitialization(ImageLoader::LinkContext const&, unsigned int, char const*, ImageLoader::InitializerTimingList&, ImageLoader::UninitedUpwards&) (in /usr/lib/dyld)
==42230==    by 0x10307E440: ImageLoader::recursiveInitialization(ImageLoader::LinkContext const&, unsigned int, char const*, ImageLoader::InitializerTimingList&, ImageLoader::UninitedUpwards&) (in /usr/lib/dyld)
==42230==    by 0x10307D523: ImageLoader::processInitializers(ImageLoader::LinkContext const&, unsigned int, ImageLoader::InitializerTimingList&, ImageLoader::UninitedUpwards&) (in /usr/lib/dyld)
==42230==    by 0x10307D5B8: ImageLoader::runInitializers(ImageLoader::LinkContext const&, ImageLoader::InitializerTimingList&) (in /usr/lib/dyld)
==42230==  Address 0x1078f465c is on thread 1's stack
==42230==  in frame #2, created by task_set_special_port (???:)
==42230==
--42230-- UNKNOWN mach_msg unhandled MACH_SEND_TRAILER option
--42230-- UNKNOWN mach_msg unhandled MACH_SEND_TRAILER option (repeated 2 times)
--42230-- UNKNOWN mach_msg unhandled MACH_SEND_TRAILER option (repeated 4 times)
==42230== Thread 2:
==42230== Invalid read of size 4
==42230==    at 0x106CE7899: _pthread_body (in /usr/lib/system/libsystem_pthread.dylib)
==42230==    by 0x106CE7886: _pthread_start (in /usr/lib/system/libsystem_pthread.dylib)
==42230==    by 0x106CE708C: thread_start (in /usr/lib/system/libsystem_pthread.dylib)
==42230==  Address 0x18 is not stack'd, malloc'd or (recently) free'd
==42230==
==42230==
==42230== Process terminating with default action of signal 11 (SIGSEGV)
==42230==  Access not within mapped region at address 0x18
==42230==    at 0x106CE7899: _pthread_body (in /usr/lib/system/libsystem_pthread.dylib)
==42230==    by 0x106CE7886: _pthread_start (in /usr/lib/system/libsystem_pthread.dylib)
==42230==    by 0x106CE708C: thread_start (in /usr/lib/system/libsystem_pthread.dylib)
==42230==  If you believe this happened as a result of a stack
==42230==  overflow in your program's main thread (unlikely but
==42230==  possible), you can try to increase the size of the
==42230==  main thread stack using the --main-stacksize= flag.
==42230==  The main thread stack size used in this run was 8388608.
==42230==
==42230== HEAP SUMMARY:
==42230==     in use at exit: 1,301,583 bytes in 807 blocks
==42230==   total heap usage: 1,751 allocs, 944 frees, 2,460,469 bytes allocated
==42230==
==42230== LEAK SUMMARY:
==42230==    definitely lost: 7,112 bytes in 96 blocks
==42230==    indirectly lost: 1,074,264 bytes in 86 blocks
==42230==      possibly lost: 3,784 bytes in 108 blocks
==42230==    still reachable: 62,761 bytes in 219 blocks
==42230==         suppressed: 153,662 bytes in 298 blocks
==42230== Rerun with --leak-check=full to see details of leaked memory
==42230==
==42230== For counts of detected and suppressed errors, rerun with: -v
==42230== Use --track-origins=yes to see where uninitialised values come from
==42230== ERROR SUMMARY: 2 errors from 2 contexts (suppressed: 28 from 4)
Segmentation fault: 11

Putting my printfs in the right place I was able to get it to print the whole of the startup game's main.js to the terminal so it is finding and opening the file - it's just when it tries to evaluate it that everything goes wrong.
Title: Re: miniSphere 4.8.4
Post by: Fat Cerberus on August 31, 2017, 04:02:43 pm
This is the cause of the segfault:
Code: [Select]
Invalid read of size 4
==42230==    at 0x106CE7899: _pthread_body (in /usr/lib/system/libsystem_pthread.dylib)
==42230==    by 0x106CE7886: _pthread_start (in /usr/lib/system/libsystem_pthread.dylib)
==42230==    by 0x106CE708C: thread_start (in /usr/lib/system/libsystem_pthread.dylib)
==42230==  Address 0x18 is not stack'd, malloc'd or (recently) free'd

Note 0x18 is a very low address - this indicates a null pointer dereference.  It's also happening off the main thread.  Hmm...
Title: Re: miniSphere 4.8.4
Post by: Rhuan on August 31, 2017, 04:08:08 pm
I tried a few other things:

1. replacing main.js with a blank file -> no segfault, minisphere opens and closes

2. replacing main,js with:
Code: [Select]
function game() {}
-> minisphere opens, the console tells me it evals the script then tries to call function game then it closes down with no segfault.

Title: Re: miniSphere 4.8.4
Post by: Fat Cerberus on August 31, 2017, 04:13:34 pm
I think there's a buffer overflow somewhere.  After screwing around with valgrind some more (you need to build CC with --valgrind on Linux apparently to get it to work), I noticed this:

Code: [Select]
Invalid read of size 4
==6756==    at 0x412079: lstr_from_wide (lstring.c:398)
==6756==    by 0x40E9A5: jsal_get_lstring (jsal.c:537)
==6756==    by 0x40EC40: jsal_get_string (jsal.c:613)
==6756==    by 0x408E27: main (main.c:314)
==6756==  Address 0x17310000 is in a --- anonymous segment

"Anonymous segment", from my cursory research, basically means "JITted code".

Pretty sure my code is buggy somewhere, it's just that Windows is a lot more forgiving of bad memory accesses.  The engine does segfault for me on occasion while calling JS functions, for what it's worth.
Title: Re: miniSphere 4.8.4
Post by: Rhuan on August 31, 2017, 04:19:16 pm
I was using the testbuild of CC, I'll see if the release build performs differently.
Title: Re: miniSphere 4.8.4
Post by: Eggbertx on August 31, 2017, 04:37:37 pm
This is a 4.8.4 build or...?

Yes, this happens whether I directly pull it or download the source code of the 4.8.4 release.
Title: Re: miniSphere 4.8.4
Post by: Rhuan on August 31, 2017, 04:38:09 pm
No improvement for using the release build.

One thing I've noted though, I've checked back through a lot of my errors and normally any segfault follows a longjump - i.e. the error seems to be in the inbuilt error handling - though obviously the error handling shouldn't be triggering in the first place as I'm giving it legit JS to read.

I've tested with adding and removing lines form the JS. And it seems that our crashes are triggered by calling font#drawText()

Title: Re: miniSphere 4.8.4
Post by: Fat Cerberus on August 31, 2017, 04:59:27 pm
What is sizeof(wchar_t) on macOS?  On Windows it's 2, on Linux it's 4.  I'm pretty sure that's why I can't get it to work in Linux, at least, CC uses UTF-16 where a widechar is 2 bytes.
Title: Re: miniSphere 4.8.4
Post by: Rhuan on August 31, 2017, 05:05:00 pm
What is sizeof(wchar_t) on macOS?  On Windows it's 2, on Linux it's 4.  I'm pretty sure that's why I can't get it to work in Linux, at least, CC uses UTF-16 where a widechar is 2 bytes.
It's 4 on macOS as well.
Title: Re: miniSphere 4.8.4
Post by: Fat Cerberus on August 31, 2017, 05:26:32 pm
Try checking out the latest build from the chakra-js branch.  I got it up on Linux.
Title: Re: miniSphere 4.8.4
Post by: Rhuan on August 31, 2017, 05:32:23 pm
Have we both just done the same thing?

I added:
typedef __CHAR16_TYPE__ char16_t;
to the top of one of your headers and then replaced all uses of wchar_t with char16_t (this definition is apparently meant to be in a standard header called uchar.h but that standard header doesn't appear to come with macos)

Having done this the startup game would then load with CC miniSphere. Unfortunately the ExecuteGame function doesn't work (I've tried running a different game by putting it in the startup folder and it gets somewhere whereas trying to call it from the startup game gives a segfault)

Found a different issue... :( Chakra Core doesn't seem to implement the TextDecoder object which causes a bit of a problem for the "standard" v2 way of reading binary data. -> apparently it's a forthcoming feature: https://wpdev.uservoice.com/forums/257854-microsoft-edge-developer/suggestions/6558040-support-the-encoding-api#{toggle_previous_statuses}
Title: Re: miniSphere 4.8.4
Post by: Fat Cerberus on August 31, 2017, 05:47:04 pm
Pretty much the same fix, I just used uint16_t instead.

TextDecoder I can fix, I'll just have to pull the original implementation from version history (I'll say it again: Git is awesome).  I implemented it in miniSphere *before* I implemented in Duktape, luckily.
Title: Re: miniSphere 4.8.4
Post by: Rhuan on August 31, 2017, 05:51:52 pm
Pretty much the same fix, I just used uint16_t instead.

TextDecoder I can fix, I'll just have to pull the original implementation from version history (I'll say it again: Git is awesome).  I implemented it in miniSphere *before* I implemented in Duktape, luckily.
Good news :)

Any ideas on ExecuteGame still segfaulting? Well speciifically error message saying "one or more components failed to load" but then on clicking ok it gives a segfault.

Also I absolutely love the backtrace for the script error screen now - that's new isn't it?
Title: Re: miniSphere 4.8.4
Post by: Fat Cerberus on August 31, 2017, 06:20:31 pm
I'll look into that segfault, it's possible Chakra doesn't like the runtime handle being disposed and then recreated.

As for the backtrace, it was kind of out of necessity since Chakra doesn't provide .fileName and .lineNumber for errors, so I just had it print the stack.  It's mostly nice to have, although it looks terrible at low resolution since it doesn't all fit and scrolls off the screen.
Title: Re: miniSphere 4.8.4
Post by: Fat Cerberus on August 31, 2017, 08:14:59 pm
@Eggbert: No luck reproducing your crash on Ubuntu (which is Debian-based). :(
Title: Re: miniSphere 4.8.4
Post by: Eggbertx on August 31, 2017, 10:46:20 pm
What about a Debian VM?

Edit: I just checked and with a basic Debian installation, just the base system, git, build-essential, liballegro5-dev, and libmng-dev, I still have the same result.
Title: Re: miniSphere 4.8.4
Post by: Fat Cerberus on September 01, 2017, 02:08:41 am
Out of curiosity I tried Radnen's speed test from his Sphere-SFML testbed.  Here's a speed comparison of for loops in miniSphere 4.8.4 (Duktape 2.1.1) vs miniSphere-CC (Chakra 1.7.1):
https://github.com/Radnen/sphere-sfml/blob/master/Engine/Engine/startup/scripts/testsuite.js

Code: [Select]
mS 5 'chakra-js'
----------------

the pig:812
manual abort

traditional for: 0.04ms
For i++ in Check: 5.4ms
For i++ in Step: 4.9ms
For i++ in Body: 5.1ms
For i += 1 in Step: 4.8ms
For i += 1 in Body: 4.8ms
For i += 2 in Step: 2.2ms
For i++ in Body and Step: 3.6ms
While i++ in Step: 5.1ms
While i++ in Body: 4.5ms


mS 4.8.4
--------

@/scripts/testsuite.js:0
manual abort

traditional for: 0ms
For i++ in Check: 185.6ms
For i++ in Step: 184.6ms
For i++ in Body: 184.2ms
For i += 1 in Step: 190.1ms
For i += 1 in Body: 188.8ms
For i += 2 in Step: 94.4ms
For i++ in Body and Step: 131.9ms
While i++ in Step: 157.6ms
While i++ in Body: 158.1ms
Title: Re: miniSphere 4.8.4
Post by: Fat Cerberus on September 01, 2017, 02:51:16 pm
Pretty much everything is up and running now, now I just have to get SSj working again.  I started playing with Chakra's debugger API, it's kind of clunky but gets the job done. :P
Title: Re: miniSphere 4.8.4
Post by: Rhuan on September 01, 2017, 03:49:04 pm
I thought I'd do some speed tests too, the below are high count speed tests so CC calls on its optimising JIT.

I also tried some low count ones to avoid triggering the optimising JIT; interestingly the speed of certain operations under CC swaps when the optimising JIT kicks in (such as Math.floor getting faster than |0).

Anyway Duktape doesn't measure up well at all...

I can't easily test my mapengine code in it's current form with Chakra without TextDecoder so will largely be sticking to Duktape for now but it's nice to see the speed and other features that are coming; great work.
Title: Re: miniSphere 4.8.4
Post by: Fat Cerberus on September 01, 2017, 03:58:31 pm
I'll get TextDecoder back up soon; SSj is taking priority because I rely on it pretty heavily myself; being able to debug Sphere games is awesome. :D
Title: Re: miniSphere 4.8.4
Post by: Fat Cerberus on September 01, 2017, 06:58:46 pm
Math.floor() being faster with JIT is interesting but ultimately not too surprising.  x | 0 involves at least a float->integer conversion (expensive) and possibly a bitwise OR, although that second one can be safely optimized out since it's a no-op.  If the JIT knows that Math.floor() hasn't been replaced it can compile it to a raw machine code equivalent, like the C compiler does with things like memset, etc.

Without the JIT Math.floor() becomes slower than |0 probably because it involves an actual function call.

But yeah, those results are phenomenal.  Itll be nice to have a modern Sphere with a blazing fast JS engine, that, the native ES6 support and the v2 API will finally bring the engine into the 21st century. :P
Title: Re: miniSphere 4.8.4
Post by: Rhuan on September 01, 2017, 07:14:15 pm
Stepping it up to 100,000,000 of each operation had some odd impacts, possibly impacted by GC slowing it down or something.

But hey still fun to see.

(I also tried taking it to 1,000,000,000 of each operation but it resulted in a segfault on the first one, I guess creating a 3.8gb array of floats was too much for it which TBH doesn't surprise me)
Title: Re: miniSphere 4.8.4
Post by: Eggbertx on September 01, 2017, 11:59:44 pm
I'm getting a (slightly) less obnoxious error message now when building the chakra-js branch in Deian
Code: [Select]
minisphere: malloc.c:2883: mremap_chunk: Assertion `((size + offset) & (GLRO (dl_pagesize) - 1)) == 0' failed.
Aborted
Title: Re: miniSphere 4.8.4
Post by: Fat Cerberus on September 02, 2017, 09:51:40 am
I'll have to set up a Debian VM at some point to test.  Then  I can run it through gdb and/or valgrind to see where the crash is.

SSj is in theory not too hard to get working again--I just have to implement Duktape's debugger protocol in miniSphere.  I already have the code to encode and decode messages for it, no sense reinventing the wheel.
Title: Re: miniSphere 4.8.4
Post by: Fat Cerberus on September 03, 2017, 01:54:49 am
Yet another Dragon Ball reference sneaks in; the new communication protocol for SSj is called the "Ki protocol".
http://dragonball.wikia.com/wiki/Ki_blast

I'm basing it on the Duktape dvalue protocol, since I already have the code written and there's nothing fundamentally wrong with it that would justify a complete rewrite of SSj's communication code.  As for miniSphere, every once in a while the engine crashes while calling into a script and I'm not sure why.  The only thing I can think of is something is getting prematurely garbage-collected because of a bug in JSAL, but I haven't found anything yet.
Title: Re: miniSphere 4.8.4
Post by: Chad Zechs on September 03, 2017, 01:12:26 pm
With all the Dragon Ball references, I'm just waiting for miniSphere to become named like Spheerus or Spheron the Eternal Coding.
Title: Re: miniSphere 4.8.4
Post by: Fat Cerberus on September 03, 2017, 01:45:25 pm
Proof of concept:
Code: [Select]
C:\src\spectacles-i>cell && ssj dist
Cell X.X.X Sphere packaging compiler (x64)
the JavaScript-powered build engine for Sphere
(c) 2015-2017 Fat Cerberus

setting up Cellscript environment...
evaluating '$/Cellscript.mjs'...
   W: no existing files match 'lib/*.js'
building targets...
   installing '@/scripts/main.js'...
cleaning up old build artifacts...
writing Sphere manifest files...
0 error(s), 1 warning(s).
SSj X.X.X Sphere JavaScript debugger (x64)
the powerful symbolic JS debugger for Sphere
(c) 2015-2017 Fat Cerberus

starting 'C:/src/spectacles-i/dist/'... OK.
connecting to 127.0.0.1:1208... OK.
establishing communication... OK.
querying target... OK.
    game: Spectacles: Bruce's Story
    author: Fat Cerberus

=> # 0: function() at @/scripts/main.js:6
6 RequireSystemScript('persist.js');

@/scripts/main.js:6 function()
(ssj) l
      1 /***
      2  * Specs Engine v6: Spectacles Saga Game Engine
      3   *           Copyright (c) 2017 Power-Command
      4 ***/
      5
=>    6 RequireSystemScript('persist.js');
      7
      8 const from     = require('from'),
      9       Console  = require('console'),
     10       Delegate = require('delegate'),

@/scripts/main.js:6 function()
(ssj) c
=> # 0: function() at @/scripts/gameDef/statuses.js:497
497                     debugger;

@/scripts/gameDef/statuses.js:497 function()
(ssj) l
    492         // cycle.
    493         specsAura: {
    494                 name: "Specs Aura",
    495                 tags: [ 'special' ],
    496                 beginCycle: function(unit, eventData) {
=>  497                         debugger;
    498                         var vit = Game.math.statValue(unit.battlerInfo.baseStats.vit, unit.battlerInfo.level);
    499                         unit.heal(0.25 * vit, [ 'specs' ]);
    500                 }
    501         },

@/scripts/gameDef/statuses.js:497 function()
(ssj) _
Title: Re: miniSphere 4.8.4
Post by: Eggbertx on September 03, 2017, 02:42:02 pm
Spheerus the (Code) Destroyer
Title: Re: miniSphere 4.8.4
Post by: Fat Cerberus on September 03, 2017, 04:51:22 pm
That could be a good name for a minifier, if nothing else. :)
Title: Re: miniSphere 4.8.4
Post by: Rhuan on September 03, 2017, 05:03:43 pm
That could be a good name for a minifier, if nothing else. :)
What I don't know is how Pegasus is related to DBZ.

Also what about adding some Yu Yu Hakusho references?

Side note, how do you like posts? I don't seem to  have the option to do it.
Title: Re: miniSphere 4.8.4
Post by: Fat Cerberus on September 03, 2017, 05:05:52 pm
The Like button should be next to the quote button, and Reply is under "More" right next to that.  At least on the default theme.

As for Pegasus, the name predates miniSphere.  I didn't come up with that one.
Title: Re: miniSphere 4.8.4
Post by: Fat Cerberus on September 03, 2017, 07:34:13 pm
Interestingly, Chakra seems to have something called "time travel debugging" that I guess lets you step backwards through your code to retrace the cause of a bug.  It's listed as experimental, but that's pretty neat.  That would be an awesome feature for SSj to have.
Title: Re: miniSphere 4.8.4
Post by: Fat Cerberus on September 04, 2017, 10:25:41 am
I'm still getting an occasional segfault in miniSphere, so I compiled a debug build of ChakraCore and ran miniSphere with it.  I keep seeing an assertion here:
https://github.com/Microsoft/ChakraCore/blob/v1.7.1/lib/Common/Memory/Recycler.inl#L59

If I continue execution from that point, I continue to get assert failures in various random locations throughout the GC code.  So I'm apparently doing something bad and confusing the garbage collector, and every once in a while it causes a hard crash.

EDIT: It seems like I'm not supposed to be calling into the JS engine at all during a finalizer call.  This will require me to change how my finalizers work, since Duktape treated them just like any other JS function call.

I hope CC isn't calling finalizers on a different thread, because my objects are internally reference counted and that would lead to very ugly race conditions.
Title: Re: miniSphere 4.8.4
Post by: Rhuan on September 04, 2017, 01:32:28 pm
I hope CC isn't calling finalizers on a different thread, because my objects are internally reference counted and that would lead to very ugly race conditions.
I think that there is a high chance that that is exactly what it's doing as one of CC's "features" is that it runs it's GC in a separate thread to avoid slowing down the execution thread, I'd imagine that the GC thread calls the finalisers.

If editing miniSphere to not have a problem with finalisers being off thread is a problem an alternative may be to edit the CC header CommonDefines.h prior to compiling CC there's a flag in there called ENABLE_CONCURRENT_GC, at a glance it looks like disabling this will result in the GC being run on the main thread, bad for performance but may fix the problem.
Title: Re: miniSphere 4.8.4
Post by: Fat Cerberus on September 04, 2017, 02:56:23 pm
Thinking about the scenarios a bit, it might not be a problem.  There are a few possibilities:


In general, the way the engine is designed, objects exposed to JS don't change hands internally unless JS code actually does something to trigger it.  And a refcount update itself is atomic.  So the off-thread finalization should actually be fine.

The crash seems to be caused by me creating new JS values during a finalizer call, which is a no-no.  I should be able to fix it easily enough.
Title: Re: miniSphere 4.8.4
Post by: Fat Cerberus on September 05, 2017, 12:48:50 am
Okay, I rewrote all the finalizers to take a native pointer directly instead of trying to read it from the JS object being finalized.  That seems to have fixed the random crashes, so I guess that was it.  Even better, finalizers now get called on shutdown again, so hopefully Valgrind will stop complaining about me leaking crazy amounts of memory.

One surefire way I found to trigger the crash in a hurry was to run a game that calls lots of Prim functions every frame.  That creates a ton of GC stress because it constantly creates temporary shapes which get drawn and then immediately go out of scope.  The kh2bar demo in particular tended to trigger it, as well as the Spectacles battle engine sooner or later because the battle screen includes a kh2bar.  I'll keep testing naturally, but the engine seems to be fully stable now. :sunglasses:

Progress report on SSj/SSj Blue:
Title: Re: miniSphere 4.8.4
Post by: Fat Cerberus on September 05, 2017, 11:13:06 am
Hm, weird.  On Linux, asprintf() exists but requires you to define _GNU_SOURCE (i.e. it's nonstandard, even in POSIX).  On OS X (and presumably iOS) it's provided by default.  Windows doesn't have it at all.  So it should be enough just to exclude my custom implementation if __APPLE__ is defined.
Title: Re: miniSphere 4.8.4
Post by: Fat Cerberus on September 05, 2017, 02:38:50 pm
SSj is slowly coming back online:
Code: [Select]
D:\src\spectacles-i>ssj dist
SSj X.X.X Sphere JavaScript debugger (x64)
the powerful symbolic JS debugger for Sphere
(c) 2015-2017 Fat Cerberus

starting 'D:/src/spectacles-i/dist/'... OK.
connecting to 127.0.0.1:1208... OK.
establishing communication... OK.
querying target... OK.
    game: Spectacles: Bruce's Story
    author: Fat Cerberus

=> # 0: function() at @/scripts/main.js:6
6 RequireSystemScript('persist.js');

@/scripts/main.js:6 function()
(ssj) c
throw: Object doesn't support property or method 'maskPerson'
    at @/scripts/maps/Testville.js:141
=> # 0: function() at @/scripts/maps/Testville.js:141
141                                     new Scene()

@/scripts/maps/Testville.js:141 function()
(ssj) l
    136                                 }
    137                                 if (!IsCameraAttached()) {  // it seems Scott got eaten...
    138                                         var session = persist.getWorld().session;
    139                                         session.party.remove('scott');
    140                                         session.party.add('maggie', 100);
=>  141                                         new Scene()
    142                                                 .fork()
    143                                                         .maskPerson('maggie', CreateColor(0, 0, 0, 0), 8)
    144                                                         .setSprite('maggie', 'battlers/maggie_hippo.rss')
    145                                                         .maskPerson('maggie', CreateColor(255, 255, 255, 255), 8)

@/scripts/maps/Testville.js:141 function()
(ssj) bt
=> # 0: function() at @/scripts/maps/Testville.js:141
   # 1: function() at #/game_modules/thread.js:195
   # 2: function() at #/runtime/from.js:267
   # 3: function() at #/runtime/from.js:706
   # 4: function() at #/runtime/from.js:101
   # 5: function() at #/game_modules/thread.js:187
   # 6: function() at @/scripts/main.js:60

@/scripts/maps/Testville.js:141 function()
(ssj) _
Title: Re: miniSphere 4.8.4
Post by: Fat Cerberus on September 05, 2017, 07:43:53 pm
By the way, that source code listing... that works even for SPK-packaged games... even if you don't have the original source available.  SSj downloads the code directly from miniSphere.  That's a big reason I wanted native ES6 support, actually.  Otherwise you just see the ugly transpiled code in the debugger. :P
Title: Re: miniSphere 4.8.4
Post by: Fat Cerberus on September 06, 2017, 01:52:42 am
@Rhuan The TextDecoder and TextEncoder objects have made their glorious return!  Now you can test your map engine with a CC-powered Sphere build. :smiley_cat:
Title: Re: miniSphere 4.8.4
Post by: Rhuan on September 06, 2017, 07:27:49 pm
So I downloaded the latest copy of the miniSphere ChakraCore branch.

1. I initially tried building it linked against the bleeding edge of chakracore -> segfault (malloc with invalid address) the moment it tried to run a script.

2. I built it linked against an older version of chakracore - not sure what version -> it starts BUT:
a) attempting to load another game from the startup game resulted in "one or more components failed to load miniSphere cannot continue in this state"
b) running my map engine test by opening the manifest with minisphere it ran BUT I got a segfault when I quit (EXC_BAD_ACCESS - it looks like an object being accessed after it was freed

I'll try testing against the 1.71 release of CC instead of using the bleeding edge and see if there's a difference
Title: Re: miniSphere 4.8.4
Post by: Fat Cerberus on September 06, 2017, 07:33:01 pm
I had problems a few days ago when I tried a bleeding-edge CC build.  It seems to be in a state of flux right now, best to just git checkout v1.7.1 and build that, or use the official CC 1.7.1 binaries from MS (this is what's in dep/ for Linux).

ExecuteGame() is still buggy, I think.  Once I get SSJ stabilized (breakpoints work now!) I'll look into that next.
Title: Re: miniSphere 4.8.4
Post by: Rhuan on September 06, 2017, 07:58:58 pm
Ok, so the segfault on exit was seemingly linked to v1 file output functions, the segfault appears to occur when trying to write data out not when closing down removing the v1 file writing got rid of the segfault.

BUT attempting to use v2 file output I get no output, I'm trying to just do:
Code: [Select]
FS.write("~/output.txt","average m_r = " + avgs[0] + "\n average s_r = " + avgs[1] + "\n average m_u = " + avgs[2] + "\n average s_u = " + avgs[3]);
It takes a very long time to run (measurable amount of time which is weird for writing such a short string) and seemingly does nothing - no output.txt exists afterwards.
Title: Re: miniSphere 4.8.4
Post by: Fat Cerberus on September 06, 2017, 08:04:16 pm
There's a bug with v1 ByteArrays - trying to access them usually causes a crash.  The problem is that they're implemented via a Proxy object and CC won't let me read the host pointer through the Proxy.  I need to refactor it to make it work again, ByteArray could probably be implemented as a subclass of Uint8Array now.

Not sure what's going on with FS.writeFile() but I'll look into it.
Title: Re: miniSphere 4.8.4
Post by: Rhuan on September 06, 2017, 08:36:46 pm
I'm afraid I found another error, after using a custom shader you can't use the Font#drawText function or at least it seemingly inherits the shader and some uniforms from an unrelated model I've been drawing (which results in the text not drawing visibly).
Title: Re: miniSphere 4.8.4
Post by: Fat Cerberus on September 06, 2017, 09:00:55 pm
That would be the lazy shader switches at work.  I'll fix that, but in the meantime adding:
Code: (C) [Select]
shader_use(galileo_shader(), false);

right before this line:
https://github.com/fatcerberus/minisphere/blob/dd31133b8828bd9690a145d7da62aad978f82430/src/minisphere/pegasus.c#L2157

should do the trick.
Title: Re: miniSphere 4.8.4
Post by: Fat Cerberus on September 06, 2017, 09:49:30 pm
@Rhuan I tried the following test:
Code: (JS) [Select]
FS.writeFile('~/812.txt', "The pig ate everyone at 8:12!\n");

It successfully wrote the file with no noticeable delays.  And if I try that from a game without a saveID defined, then I get:

Code: [Select]
JavaScript exception!

ReferenceError: no save ID defined
   at Showcase (:32:3)

So it shouldn't be silently failing, that's kind of weird.
Title: Re: miniSphere 4.8.4
Post by: Fat Cerberus on September 07, 2017, 09:46:54 am
Debugger bugs, what fun.  See if you can spot what's wrong with this picture (besides game() being called an anonymous function...) :P

Code: [Select]
D:\src\spectacles-i>ssj dist
SSJ X.X.X Sphere JavaScript debugger (x64)
the powerful symbolic JS debugger for Sphere
(c) 2015-2017 Fat Cerberus

starting 'D:/src/spectacles-i/dist/'... OK.
connecting to 127.0.0.1:1208... OK.
establishing communication... OK.
querying target... OK.
    game: Spectacles: Bruce's Story
    author: Fat Cerberus

=> # 0: [anonymous function] at @/scripts/main.js:6
6 RequireSystemScript('persist.js');

@/scripts/main.js:6 [anonymous function]
(ssj) c
UNCAUGHT: Error: The pig ate everything at 8:12
   at Anonymous function (@/scripts/main.js:45:12)
   at game (@/scripts/main.js:45:4)
=> # 0: [anonymous function] at @/scripts/main.js:49
49      Console.initialize({ hotKey: Key.Tilde });

@/scripts/main.js:49 [anonymous function]
(ssj) l 20
     39
     40         persist.init();
     41
     42         var evens = from([ 1, 2, 3, 4, 5, 6 ])
     43                 .where(it => it % 2 == 0);
     44         for (let even of evens) {
     45                 (() => { throw new Error("The pig ate everything at 8:12"); })();
     46                 SSj.log(even);
     47         }
     48
=>   49         Console.initialize({ hotKey: Key.Tilde });
     50         Console.defineObject('yap', null, {
     51                 'on': function() {
     52                         Sphere.Game.disableTalking = false;
     53                         Console.log("oh, yappy times are here again...");
     54                 },
     55                 'off': function() {
     56                         Sphere.Game.disableTalking = true;
     57                         Console.log("the yappy times are OVER!");
     58                 },

@/scripts/main.js:49 [anonymous function]
(ssj) bt
=> # 0: [anonymous function] at @/scripts/main.js:49

@/scripts/main.js:49 [anonymous function]
(ssj) _
Title: Re: miniSphere 4.8.4
Post by: Fat Cerberus on September 08, 2017, 01:21:06 pm
I've decided to release this as miniSphere 5.0 after all.  I've traditionally done a major version increase whenever a change represents a major paradigm shift:


Since introducing ChakraCore is not just a performance boost but also introduces native support for ES6, allowing games to once again be created without the need for Cell, that feels like enough of a paradigm shift to warrant a version bump.

I'll release the first public beta of miniSphere 5.0 in a week or so.

Since I did promise an API freeze for v5.0, I'm still going to try to achieve that.  We'll see what happens on that front.
Title: Re: miniSphere 4.8.4
Post by: Fat Cerberus on September 09, 2017, 12:50:24 am
SSj is about 95% feature-complete now.  The ChakraCore experiment was more successful than I ever imagined, not only do I get all ES6 syntax and native modules (which is a big deal--very few engines support these!), I was even able to more or less replicate all the functionality I had with the (surprisingly comprehensive!) Duktape debugger.  The engine is stable enough now that I'm thinking I'll lock it in and merge all the CC stuff into the trunk.  There's still a few kinks to work out before miniSphere 5.0.0 is released (like figuring out how to static-link CC), but overall I consider what's here good enough to be merged into the main codebase.

Once I do the merge I'll start looking at things in more detail, like figuring out @Eggbertx crashes on Debian.  I also need to clean up the codebase at some point, because I did a lot of hacky retrofitting to avoid touching a lot of code.  Changing JS engines was already enough of an upheaval that I didn't want to risk introducing even more bugs by indiscriminately editing things. :P
Title: Re: miniSphere 4.8.4
Post by: Fat Cerberus on September 09, 2017, 10:50:06 am
@Eggbertx I can't reproduce your crash on Debian 9.  After struggling with it for a bit (Debian doesn't install or set up sudo by default, which was surprising), I finally got it set up enough to compile and run miniSphere and... it worked fine.  The setup is pretty basic, I just installed:

Title: Re: miniSphere 4.8.4
Post by: Eggbertx on September 09, 2017, 11:42:57 am
I reinstalled Debian again just to be sure that I hadn't installed any extra packages that could be interfering and installed LXDE, and after the CD installation was done, I booted into it, logged in, and ran apt install git liballegro5-dev libmng-dev, which installed gcc and all the dependencies. Then I cloned the minisphere, ran make and make install, then I ran minisphere and still had the same issue when I exited out of that.
Title: Re: miniSphere 4.8.4
Post by: Fat Cerberus on September 09, 2017, 11:46:20 am
Try this:
Install valgrind
Build miniSphere using "make clean all CFLAGS=-g"
sudo make install
valgrind minisphere

Then post the output.
Title: Re: miniSphere 4.8.4
Post by: Eggbertx on September 09, 2017, 12:09:52 pm
Code: [Select]
==2047== Memcheck, a memory error detector
==2047== Copyright (C) 2002-2015, and GNU GPL'd, by Julian Seward et al.
==2047== Using Valgrind-3.12.0.SVN and LibVEX; rerun with -h for copyright info
==2047== Command: minisphere
==2047==
Out of Memory
==2047==
==2047== Process terminating with default action of signal 6 (SIGABRT)
==2047==    at 0x8EC3FCF: raise (raise.c:51)
==2047==    by 0x8EC53F9: abort (abort.c:89)
==2047==    by 0x60EC165: _GLOBAL__sub_I_RecyclerWriteBarrierManager.cpp (in /usr/lib/libChakraCore.so)
==2047==    by 0x400F8A9: call_init.part.0 (dl-init.c:72)
==2047==    by 0x400F9BA: call_init (dl-init.c:30)
==2047==    by 0x400F9BA: _dl_init (dl-init.c:120)
==2047==    by 0x4000C59: ??? (in /lib/x86_64-linux-gnu/ld-2.24.so)
==2047==
==2047== HEAP SUMMARY:
==2047==     in use at exit: 86,856 bytes in 262 blocks
==2047==   total heap usage: 312 allocs, 50 frees, 96,876 bytes allocated
==2047==
==2047== LEAK SUMMARY:
==2047==    definitely lost: 0 bytes in 0 blocks
==2047==    indirectly lost: 0 bytes in 0 blocks
==2047==      possibly lost: 1,800 bytes in 19 blocks
==2047==    still reachable: 85,056 bytes in 243 blocks
==2047==                       of which reachable via heuristic:
==2047==                         newarray           : 1,536 bytes in 16 blocks
==2047==         suppressed: 0 bytes in 0 blocks
==2047== Rerun with --leak-check=full to see details of leaked memory
==2047==
==2047== For counts of detected and suppressed errors, rerun with: -v
==2047== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)
Killed
Title: Re: miniSphere 4.8.4
Post by: Fat Cerberus on September 09, 2017, 12:45:17 pm
Right, I forgot Chakra and Valgrind don't get along without some coercing (read: building CC manually).  Can you do the same thing with 4.8.4?  You said the Duktape engines were crashing too.
Title: Re: miniSphere 4.8.4
Post by: Rhuan on September 09, 2017, 06:45:37 pm
I've decided to release this as miniSphere 5.0 after all.  I've traditionally done a major version increase whenever a change represents a major paradigm shift:

  • 1.0: ~90% Sphere 1.x compatibility
  • 2.0: Cell, SphereFS, s2gm
  • 3.0: SSj, Linux support
  • 4.0: Sphere v2 API

Since introducing ChakraCore is not just a performance boost but also introduces native support for ES6, allowing games to once again be created without the need for Cell, that feels like enough of a paradigm shift to warrant a version bump.

I'll release the first public beta of miniSphere 5.0 in a week or so.

Since I did promise an API freeze for v5.0, I'm still going to try to achieve that.  We'll see what happens on that front.
I was hoping I'd have the map engine ready by the time you were at version 5 but I've spent too much time working this week and haven't made much progress at all - will see what I can do tomorrow, out of time for today. :(

Well done on getting miniSphere running on Chakra though, really good work.


Though I'm still trying to think what would be a good use of all of that power - other than doing 10,000,000s of unnecessary random sums of course :p
Title: Re: miniSphere 4.8.5
Post by: DaVince on September 10, 2017, 06:15:26 am
I finally got some time to compile miniSphere today. It works when I sudo make install, but locally it just goes:

Code: [Select]
./minisphere: error while loading shared libraries: libChakraCore.so: cannot open shared object file: No such file or directory

The startup game runs properly if I do a global install.

Edit: running a game (Aquatis) from the startup fails with a little popup window with a grammar error though:
Code: [Select]
Engine initialized failed.
The console just segfaults at this point.
Title: Re: miniSphere 4.8.5
Post by: Rhuan on September 10, 2017, 08:02:00 am
I finally got some time to compile miniSphere today. It works when I sudo make install, but locally it just goes:

Code: [Select]
./minisphere: error while loading shared libraries: libChakraCore.so: cannot open shared object file: No such file or directory

The startup game runs properly if I do a global install.

Edit: running a game (Aquatis) from the startup fails with a little popup window with a grammar error though:
Code: [Select]
Engine initialized failed.
The console just segfaults at this point.
I think it's just that Execute Game is broken in the CC version at the moment, try opening a game by opening the .sgm or .json with miniSphere.
Title: Re: miniSphere 4.8.5
Post by: Fat Cerberus on September 10, 2017, 09:12:34 am
I finally got some time to compile miniSphere today. It works when I sudo make install, but locally it just goes:

Code: [Select]
./minisphere: error while loading shared libraries: libChakraCore.so: cannot open shared object file: No such file or directory

@DaVince That's expected, unfortunately.  Unlike Windows, Linux won't just load shared libraries (Linux equivalent to a DLL) from the program directory, you have to install the .so to a known location.  And there are no ChakraCore packages for Ubuntu yet, not even a PPA.  The final release will be static linked to CC to avoid the issue.

And yeah, like @Rhuan said, ExecuteGame() is broken at the moment :(
Title: Re: miniSphere 4.8.5
Post by: Fat Cerberus on September 10, 2017, 09:41:27 am
Huh, weird.  ExecuteGame() works perfectly on Windows, I can't reproduce the crash in either a Debug or Release build.  I'll try on Linux.
Title: Re: miniSphere 4.8.5
Post by: Fat Cerberus on September 10, 2017, 10:02:54 am
Yep, segfault on Linux when calling ExecuteGame.  I fired up GDB and this is what I got:
Code: [Select]
minisphere: malloc.c:2868: mremap_chunk: Assertion `((size + offset) & (GLRO (dl_pagesize) - 1)) == 0' failed.

Looks like some sort of heap corruption.
Title: Re: miniSphere 4.8.5
Post by: DaVince on September 10, 2017, 10:04:41 am
I finally got some time to compile miniSphere today. It works when I sudo make install, but locally it just goes:

Code: [Select]
./minisphere: error while loading shared libraries: libChakraCore.so: cannot open shared object file: No such file or directory

@DaVince That's expected, unfortunately.  Unlike Windows, Linux won't just load shared libraries (Linux equivalent to a DLL) from the program directory, you have to install the .so to a known location.  And there are no ChakraCore packages for Ubuntu yet, not even a PPA.  The final release will be static linked to CC to avoid the issue.

And yeah, like @Rhuan said, ExecuteGame() is broken at the moment :(
Ah, that is too bad. Looking into it, this could be fixed (https://serverfault.com/a/402590/397047) by adding -R <path> to the make command. Perhaps this could be added to the makefile in a way too.
Title: Re: miniSphere 4.8.5
Post by: Fat Cerberus on September 10, 2017, 10:12:39 am
And well, here's the problem:
Code: [Select]
#5  0x0000000000418df5 in ensure_space (vector=0x7b42a0, min_items=33)
    at src/shared/vector.c:192
192 if (!(new_buffer = realloc(vector->buffer, new_max * vector->pitch)) && new_max > 0)
(gdb) l
187 if (min_items > vector->max_items)  // is the buffer too small?
188 new_max = min_items * 2;
189 else if (min_items < vector->max_items / 4)  // if item count drops below 1/4 of peak size, shrink the buffer
190 new_max = min_items;
191 if (new_max != vector->max_items) {
192 if (!(new_buffer = realloc(vector->buffer, new_max * vector->pitch)) && new_max > 0)
193 return false;
194 vector->buffer = new_buffer;
195 vector->max_items = new_max;
196 }
(gdb) p new_max
$1 = 66
(gdb) p vector->pitch
$2 = 140701562626560

realloc() gets asked to allocate 9 quadrillion bytes and, well, that obviously fails miserably. :P
Title: Re: miniSphere 4.8.5
Post by: Fat Cerberus on September 10, 2017, 10:42:31 am
Okay, well, the crashes themselves are fixed, but the underlying problem remains: The engine fails to reinitialize after an ExecuteGame().  I haven't figured out if that's related to ChakraCore or something else.  I can't reproduce it on Windows though, which is weird.

Oh, and I fixed the ungrammatical error message. ;)
Title: Re: miniSphere 4.8.4
Post by: Fat Cerberus on September 10, 2017, 11:56:09 am
I'm getting a (slightly) less obnoxious error message now when building the chakra-js branch in Deian
Code: [Select]
minisphere: malloc.c:2883: mremap_chunk: Assertion `((size + offset) & (GLRO (dl_pagesize) - 1)) == 0' failed.
Aborted

This sounds like the bug I just fixed (exact same error), after calling ExecuteGame() the engine fails to reinitialize but continues running anyway, eventuallly leading to a bad realloc and subsequent crash.  Does the CC-enabled build run at all?
Title: Re: miniSphere 4.8.5
Post by: Rhuan on September 10, 2017, 12:47:18 pm
I've got a feature request that may sound a bit odd...

Please can we have shapes and shaders and surfaces available in cell. I want to run:
- my map loading script in cell (it draws all the tiles on a layer to make one image for that layer)
- my rss loader (it combines the frames into an atlas)
- some similar animation code I've just been thinking of (again combine a stack of frames into an atlas)

Thoughts? Possible?
Title: Re: miniSphere 4.8.5
Post by: Fat Cerberus on September 10, 2017, 02:15:39 pm
That would mean taking a dependency on Allegro, which seems a bit bloaty for a command-line tool.  In fact having a whole rendering pipeline in a compiler seems like a lot of extra complexity to me.

That said, I did want to have some kind of image-manipulation built-in, as raw JavaScript is just not suited to it, even with ArrayBuffers.  I had a libpng-based image loader in the codebase originally (not exposed to the API), but it got blown away when I rewrote the compiler from the ground up for miniSphere 4.4.

What you describe is in fact exactly the kind of thing Cell was designed for.  It's just that I never wrote the bindings for it. ;)
Title: Re: miniSphere 4.8.5
Post by: Rhuan on September 10, 2017, 05:20:29 pm
Hmm thinking about it I'd be happy doing the image manipulation with an image object that was simply:
Code: [Select]
{
  width:num_pixels,
  height:num_pixels,
  data:array_of_pixels
}
The array could just be 1d but with each element being a color object (or equivalent - just needs the rgba values)

Don't need a 2d construct as can work fine with specified width to tell you which row you're on (could have a helper function that converts coordinates to an index location); as I'd want to combine images having a more complex interface would probably make it harder.

The key things I'd need which we don't have at the moment are methods to:
a) convert png/jpeg files to width + height + pixel array
b) save a width + height + pixel array in such a way that reading it in as an ArrayBuffer and then calling new Texture on it in miniSphere will work.

I probably could implement all of that in JS but it may be slightly painful.


EDIT: Hmm, lets see where I can get to with this starting point: https://github.com/arian/pngjs
Title: Re: miniSphere 4.8.5
Post by: Fat Cerberus on September 10, 2017, 07:59:09 pm
Ah, that is too bad. Looking into it, this could be fixed (https://serverfault.com/a/402590/397047) by adding -R <path> to the make command. Perhaps this could be added to the makefile in a way too.

Thanks for looking into that for me @DaVince, that worked great!  One just has to set the executable's RPATH to $ORIGIN and then it automatically loads dynamic libraries from the same directory as itself, just like in Windows :)
Title: Re: miniSphere 4.8.5
Post by: Fat Cerberus on September 10, 2017, 08:36:35 pm
Edit: running a game (Aquatis) from the startup fails with a little popup window with a grammar error though:
Code: [Select]
Engine initialized failed.
The console just segfaults at this point.

Interesting, I stepped through the initialization process with GDB and the second time JsCreateContext() is called it fails with a "wrong thread" error.  This makes no sense since the same thread is creating the context that created the runtime.  The only thing I can think of is that me longjmp'ing totally clear of the JS engine in order to do ExecuteGame is corrupting stuff.  Duktape didn't seem to mind me doing that, but Chakra has a lot more that can go wrong.
Title: Re: miniSphere 4.8.5
Post by: Eggbertx on September 11, 2017, 03:44:46 am
Here is valgrind's output of the 4.8.4 release
Code: [Select]
==3468== Conditional jump or move depends on uninitialised value(s)
==3468==    at 0x4C32EA6: rawmemchr (vg_replace_strmem.c:1402)
==3468==    by 0x67DA391: _IO_str_init_static_internal (strops.c:41)
==3468==    by 0x67CDB66: vsscanf (iovsscanf.c:40)
==3468==    by 0x67C82D6: sscanf (sscanf.c:32)
==3468==    by 0xF547E4A: ??? (in /usr/lib/x86_64-linux-gnu/libdrm.so.2.4.0)
==3468==    by 0xF548182: ??? (in /usr/lib/x86_64-linux-gnu/libdrm.so.2.4.0)
==3468==    by 0xF54BBF8: drmGetDevice (in /usr/lib/x86_64-linux-gnu/libdrm.so.2.4.0)
==3468==    by 0x94932D6: ??? (in /usr/lib/x86_64-linux-gnu/libGL.so.1.2.0)
==3468==    by 0x9492694: ??? (in /usr/lib/x86_64-linux-gnu/libGL.so.1.2.0)
==3468==    by 0x9466988: ??? (in /usr/lib/x86_64-linux-gnu/libGL.so.1.2.0)
==3468==    by 0x9461F10: glXQueryVersion (in /usr/lib/x86_64-linux-gnu/libGL.so.1.2.0)
==3468==    by 0x5D470B8: ??? (in /usr/lib/x86_64-linux-gnu/liballegro.so.5.2.2)
==3468==
(Now I press Escape, select Exit and...)
==3468== Invalid read of size 4
==3468==    at 0x1902DE: image_unref (image.c:272)
==3468==    by 0x1A9A64: screen_free (screen.c:145)
==3468==    by 0x112797: shutdown_engine (main.c:484)
==3468==    by 0x111BFD: main (main.c:167)
==3468==  Address 0x2a10e590 is 0 bytes inside a block of size 112 free'd
==3468==    at 0x4C2CDDB: free (vg_replace_malloc.c:530)
==3468==    by 0x190369: image_unref (image.c:282)
==3468==    by 0x1A81D2: js_Surface_finalize (pegasus.c:4095)
==3468==    by 0x149B46: duk__handle_call_inner (duk_js_call.c:1619)
==3468==    by 0x14906F: duk_handle_call_unprotected (duk_js_call.c:1149)
==3468==    by 0x1185B9: duk_call (duk_api_call.c:60)
==3468==    by 0x13A10B: duk__finalize_helper (duk_heap_finalize.c:368)
==3468==    by 0x14AAA4: duk__handle_safe_call_inner (duk_js_call.c:2131)
==3468==    by 0x14A91B: duk_handle_safe_call (duk_js_call.c:1972)
==3468==    by 0x118928: duk_safe_call (duk_api_call.c:228)
==3468==    by 0x13A18A: duk_heap_run_finalizer (duk_heap_finalize.c:430)
==3468==    by 0x139903: duk__free_run_finalizers (duk_heap_alloc.c:206)
==3468==  Block was alloc'd at
==3468==    at 0x4C2DBC5: calloc (vg_replace_malloc.c:711)
==3468==    by 0x18F787: image_new (image.c:69)
==3468==    by 0x1A98DB: screen_new (screen.c:100)
==3468==    by 0x111E9D: main (main.c:227)
==3468==
==3468== Invalid write of size 4
==3468==    at 0x1902E7: image_unref (image.c:272)
==3468==    by 0x1A9A64: screen_free (screen.c:145)
==3468==    by 0x112797: shutdown_engine (main.c:484)
==3468==    by 0x111BFD: main (main.c:167)
==3468==  Address 0x2a10e590 is 0 bytes inside a block of size 112 free'd
==3468==    at 0x4C2CDDB: free (vg_replace_malloc.c:530)
==3468==    by 0x190369: image_unref (image.c:282)
==3468==    by 0x1A81D2: js_Surface_finalize (pegasus.c:4095)
==3468==    by 0x149B46: duk__handle_call_inner (duk_js_call.c:1619)
==3468==    by 0x14906F: duk_handle_call_unprotected (duk_js_call.c:1149)
==3468==    by 0x1185B9: duk_call (duk_api_call.c:60)
==3468==    by 0x13A10B: duk__finalize_helper (duk_heap_finalize.c:368)
==3468==    by 0x14AAA4: duk__handle_safe_call_inner (duk_js_call.c:2131)
==3468==    by 0x14A91B: duk_handle_safe_call (duk_js_call.c:1972)
==3468==    by 0x118928: duk_safe_call (duk_api_call.c:228)
==3468==    by 0x13A18A: duk_heap_run_finalizer (duk_heap_finalize.c:430)
==3468==    by 0x139903: duk__free_run_finalizers (duk_heap_alloc.c:206)
==3468==  Block was alloc'd at
==3468==    at 0x4C2DBC5: calloc (vg_replace_malloc.c:711)
==3468==    by 0x18F787: image_new (image.c:69)
==3468==    by 0x1A98DB: screen_new (screen.c:100)
==3468==    by 0x111E9D: main (main.c:227)
==3468==
==3468== Invalid read of size 4
==3468==    at 0x1902ED: image_unref (image.c:272)
==3468==    by 0x1A9A64: screen_free (screen.c:145)
==3468==    by 0x112797: shutdown_engine (main.c:484)
==3468==    by 0x111BFD: main (main.c:167)
==3468==  Address 0x2a10e590 is 0 bytes inside a block of size 112 free'd
==3468==    at 0x4C2CDDB: free (vg_replace_malloc.c:530)
==3468==    by 0x190369: image_unref (image.c:282)
==3468==    by 0x1A81D2: js_Surface_finalize (pegasus.c:4095)
==3468==    by 0x149B46: duk__handle_call_inner (duk_js_call.c:1619)
==3468==    by 0x14906F: duk_handle_call_unprotected (duk_js_call.c:1149)
==3468==    by 0x1185B9: duk_call (duk_api_call.c:60)
==3468==    by 0x13A10B: duk__finalize_helper (duk_heap_finalize.c:368)
==3468==    by 0x14AAA4: duk__handle_safe_call_inner (duk_js_call.c:2131)
==3468==    by 0x14A91B: duk_handle_safe_call (duk_js_call.c:1972)
==3468==    by 0x118928: duk_safe_call (duk_api_call.c:228)
==3468==    by 0x13A18A: duk_heap_run_finalizer (duk_heap_finalize.c:430)
==3468==    by 0x139903: duk__free_run_finalizers (duk_heap_alloc.c:206)
==3468==  Block was alloc'd at
==3468==    at 0x4C2DBC5: calloc (vg_replace_malloc.c:711)
==3468==    by 0x18F787: image_new (image.c:69)
==3468==    by 0x1A98DB: screen_new (screen.c:100)
==3468==    by 0x111E9D: main (main.c:227)
==3468==
Title: Re: miniSphere 4.8.5
Post by: DaVince on September 11, 2017, 06:29:38 am
Anything I can do to help debug the issue? I guess I should be running Valgrind and compiling with debugging symbols too, huh?

Edit: by the way, I also made an #engine-dev channel on the new Discord server where we could possibly do this in a more real-time fashion. ;)
Title: Re: miniSphere 4.8.5
Post by: Fat Cerberus on September 11, 2017, 09:37:43 am
@Eggbertx Yep, I just stumbled into that one last night actually, it's a nasty double free + use-after-free caused by:
https://github.com/fatcerberus/minisphere/blob/v4.8.4/src/minisphere/pegasus.c#L626

The backbuffer is exposed in JS as screen, however it's registered into the API without calling image_ref() so once the Surface finalizer runs it frees the last reference.  Then miniSphere goes to free it again on shutdown (or switching games) and bam, crash.  Luckily it's a trivial fix.

See how useful Valgrind is?
Title: Re: miniSphere 4.8.5
Post by: Rhuan on September 11, 2017, 04:40:44 pm
Hmm thinking about it I'd be happy doing the image manipulation with an image object that was simply:
Code: [Select]
{
  width:num_pixels,
  height:num_pixels,
  data:array_of_pixels
}
The array could just be 1d but with each element being a color object (or equivalent - just needs the rgba values)

Don't need a 2d construct as can work fine with specified width to tell you which row you're on (could have a helper function that converts coordinates to an index location); as I'd want to combine images having a more complex interface would probably make it harder.

The key things I'd need which we don't have at the moment are methods to:
a) convert png/jpeg files to width + height + pixel array
b) save a width + height + pixel array in such a way that reading it in as an ArrayBuffer and then calling new Texture on it in miniSphere will work.

I probably could implement all of that in JS but it may be slightly painful.


EDIT: Hmm, lets see where I can get to with this starting point: https://github.com/arian/pngjs

To get this going I need a zlib.inflate function I can call, I've found three different zlib.js scripts but I can't get any of them to run they all need either node.js api features or browser api feactures and I can't manage to hack those requirements out of them easily; probably could if I spent a long time at it but this (i.e. cobbling together png support with pure JS) feels like excessive effort for something that could probably be more easily implemented in C.
Title: Re: miniSphere 4.8.5
Post by: Rhuan on September 11, 2017, 06:30:54 pm
Posted this on the discord but worth saying here too, for anyone building chakra core on linux/macos to test the latest version of minisphere use:
./build.sh --no-icu -j=4
Optionally you can also add --static to make it build static.

If you call the build script with no options it:
a) insists on linking dynamically to libicu (which is only needed for facilities that minisphere doesn't use)
b) builds with only one thread (so takes ages to build)
c) produces a dynamic library

Note, if you build with --static but without the --no-icu it will build a static library linked to a dynamic version of icu which seems silly to me.

If for some reason you want to include icu you can also use --embed-icu this makes the build script download icu and then it links to it statically so you don't have to worry about it. I can't find a way to fully static link without using this feature.

Title: Re: miniSphere 4.8.5
Post by: Fat Cerberus on September 11, 2017, 07:05:01 pm
Good to know about that -j option.  On Windows with MSVC it's a 2-3 minute build while Linux was taking 15-20 minutes for me and I couldn't figure out why.
Title: Re: miniSphere 4.8.5
Post by: Fat Cerberus on September 12, 2017, 01:44:02 am
Finally got breakpoints fully working in SSj.  I couldn't get it to honor breakpoints in a .mjs, JsDiagSetBreakpoint() kept returning an error code.  I thought it was a bug in ChakraCore not letting breakpoints work in modules but it turns out I was just tracking script IDs wrong (trying to set the breakpoint in the wrong script).

Only one thing still doesn't work in SSj: the examine command.  With the Duktape engines you can use it to see all properties on an object and their attributes:
Code: [Select]
#/game_modules/console.js:86 [anon]
(ssj) x Sphere.Game
ew-c  name                "Spectacles: Bruce's Story"
ew-c  version             1
ew-c  saveID              "fatCerberus.spectacles"
ew-c  author              "Fat Cerberus"
ew-c  summary             "Follow Scott Starcross in his quest to stop the Primus from destroying both worlds."
ew-c  resolution          "320x200"
ew-c  main                "@/scripts/main.js"
ew-c  disableAnimations   false
ew-c  disableBattles      false
ew-c  disableSplash       false
ew-c  disableTalking      false
ew-c  disableTitleScreen  false

I haven't implemented object inspection in JSAL yet though, so in the 5.0 alpha examine just acts like a normal eval:
Code: [Select]
#/game_modules/console.js:86 [anonymous function]
(ssj) x Sphere.Game
eval() = {...}
Title: Re: miniSphere 4.8.5
Post by: Rhuan on September 12, 2017, 03:39:15 am
I'd like to be able to build miniSphere from the command line updating an xcode project for even new release is super tedious.

Unfortunately though a few tweaks to the makefile you provide got it to build it just hangs when it opens :( - the same source built with my xcode project works (well execute game still doesn't work but otherwise it seems good).
Title: Re: miniSphere 4.8.5
Post by: Fat Cerberus on September 12, 2017, 10:51:24 am
@Rhuan I added self-path logic for macOS:
https://github.com/fatcerberus/minisphere/commit/1545ecceb0c1ec4fdcf5a3b0fff733e9263e94d5

One less thing you have to patch ;)
Title: Re: miniSphere 4.8.6
Post by: Rhuan on September 12, 2017, 08:29:13 pm
Did some more testing and I think I've identified a clear issue  in miniSphere with chakra - the api functions are just slower to execute; some are worse than others - a particular culprit is new VertexList which takes 50% longer in the chakra version which will be noticeable at run time as it's already a slow function.

In a duktape render function you could just about afford to call new VertexList 50 times per frame and you'd still have a little breathing room for something else - though not much. In a CC render function 35 calls of new VertexList is about your limit per frame.

Initially on doing this new round of tests I thought it may be particularly an issue with new VertexList but I checked some other things for comparison and the issue is more prevalent.

So as another example Rectangle from the sphere v1 api takes 4x longer in its CC implementation than in the duktape version - I note however that it's sufficiently quick that that should rarely be an issue. (new VertexList takes over 20 times as long to run as Rectangle in both versions).

Even IsAnyKeyPressed() takes over twice as long in the chakra version than in the Duktape version.

Other thoughts I think my theory about garbage collection issues earlier was totally wrong - I think it's all down to the slower api function execution times; as for why they're slower I don't know, my guess at first was it would relate to some inefficiency in the way you collect values with the various jsal_require functions but the fact that even a parameterless function like IsAnyKeyPressed was so much slower highly confused me there.

Note: all speed comparisons mentioned above were done using 20,000 repetitions or more and repeated several times
Title: Re: miniSphere 4.8.6
Post by: Fat Cerberus on September 13, 2017, 12:56:40 am
If I'm going to hazard a guess, I'd surmise that crossing the JS<->C boundary is inherently expensive.  In Duktape a native call is super cheap; in fact, other things being equal, it's probably faster to call an API function than an equivalent JS function just because there's less overhead.  Duktape has to set up a stack frame and push the parameters (and then pop them on return), but that's it, otherwise it's just a straight C function call.  A JS call involves a lot more bookkeeping including setting up a new bytecode executor instance, etc.

In an engine like Chakra the story is different: JS is a very dynamic language and optimizations rely on making educated guesses about what the code is going to do based on things like static analysis, observation of the code at runtime, what have you.  Once you call into native code though, all bets are off, the call is essentially a black box from the point of view of the engine and pretty much anything could happen.  JS values might be modified, types changed in unexpected ways, etc.

So yeah, my hypothesis is that making so many API calls causes a lot of optimizations and/or JIT'd code to be thrown away, which is expensive for the much the same reason as a branch prediction failure is expensive on a modern CPU.
Title: Re: miniSphere 4.8.6
Post by: Rhuan on September 13, 2017, 02:19:42 am
I can't see why the API calls should result in JIT'd code being thrown - from my understanding normally that should only happen if variables/parameters used within the JIT'd code unexpectedly change types.

Additionally if it was a general issue with native calls I'd expect there to be a flat increase instead different functions have different increases e.g.

Rectangle takes 4 times longer under chakra
new VertexBuffer takes 1.5 times as long under chakra
IsAnyKeyPressed takes 1.66 times as long
new Shape takes 4.5 times as long
Shape.draw() takes 2.5 times as long

And these ratios are consistent implying something specific about the internal workings of these functions that causes the time increase.

I note that in most cases these functions are still short enough that they'll rarely be a problem anyway despite the increase; there are some exceptions though such as new VertexBuffer which was just barely ok for frequent use under duktape and now really isn't under chakra.
Title: Re: miniSphere 4.8.6
Post by: Rhuan on September 13, 2017, 03:29:49 am
I've done some more detailed tests and written the results out to some tables - see below. These timings are consistent (note that new VertexBuffer is intentionally only called 20,000 times compared with the 200,000 for the others due to its comparative excessive length.

I feel like there has to be some kind of pattern underpinning this but I don't know what.

Times with ChrakaCore miniSphere.
EventTimeTakenNumbertime for 1in 8 milliseconds
shape.draw9422000000.004711,698.51
new Shape14142000000.007071,131.54
new VertexList5042200000.252131.73
Rectangle17572000000.008785910.64
IsAnyKeyPressed1752000000.0008759,142.86
CreateColor16912000000.008455946.19
Color52000000.000025320,000.00

Times with Duktape miniSphere.

EventTimeTakenNumbertime for 1in 8 milliseconds
shape.draw4202000000.00213,809.52
new Shape3192000000.0015955,015.67
new VertexList2866200000.143355.83
Rectangle4692000000.0023453,411.51
IsAnyKeyPressed912000000.00045517,582.42
CreateColor3372000000.0016854,747.77
new Color02000000n/a
Title: Re: miniSphere 4.8.6
Post by: Fat Cerberus on September 13, 2017, 09:19:27 am
new Color() is ridiculously cheaper than CreateColor()... why?
Title: Re: miniSphere 4.8.6
Post by: Fat Cerberus on September 13, 2017, 11:20:08 am
@Rhuan

Color constructor:
https://github.com/fatcerberus/minisphere/blob/master/src/minisphere/pegasus.c#L1483-L1508

CreateColor:
https://github.com/fatcerberus/minisphere/blob/master/src/minisphere/vanilla.c#L1905-L1926

Notice anything that might cause the latter to be significantly slower?
Title: Re: miniSphere 4.8.6
Post by: Fat Cerberus on September 13, 2017, 11:38:30 am
For anyone who's already downloaded 4.8.6: I just reuploaded the release because I found out I forgot to include Sphere Studio in the installer! :-X
Title: Re: miniSphere 4.8.6
Post by: Rhuan on September 13, 2017, 01:27:44 pm
@Rhuan

Color constructor:
https://github.com/fatcerberus/minisphere/blob/master/src/minisphere/pegasus.c#L1483-L1508

CreateColor:
https://github.com/fatcerberus/minisphere/blob/master/src/minisphere/vanilla.c#L1905-L1926

Notice anything that might cause the latter to be significantly slower?
i'm not really a c programmer at all though would like to learn.

My only guesses from just looking at it right now are:

Either;
a) I set up the test wrongly somehow (will recheck shortly)
Or
b) there's something expensive about getting an integer from JS - possible as natural JS numbers are double floats.
Or
c) it's the fact that create color caps the inputs to 255 hence uses several conditionals whereas new color doesn't apply checks - those checks in create color won't take long but they won't take no time

I think the key next step is to run minisphere with some kind of c profiler and find out which jsal functions it's spending time in - not quite sure how to set that up as can't loop them easily and they all take <1 millisecond.
Title: Re: miniSphere 4.8.6
Post by: Fat Cerberus on September 13, 2017, 01:56:31 pm
You might be onto something here.  `jsal_get_int()` does clamping to [INT_MIN,INT_MAX] followed by a double-to-int cast:
https://github.com/fatcerberus/minisphere/blob/834a2ce24b2db064e7768cb48805b2093da17273/src/shared/jsal.c#L553-L557

Chakra has a function `JsNumberToInt()`, I should see if using that performs better than doing the cast myself.  It's possible that Chakra is storing integers as actual ints internally, so that calling `JsNumberToDouble()` followed by an int cast involves a round-trip int->double->int, which is going to be expensive especially if you do it for every single function parameter.
Title: Re: miniSphere 4.8.6
Post by: Rhuan on September 13, 2017, 02:22:20 pm
I've checked over my test and re-run it; I'm confident that it's right and no change to the above results.
Title: Re: miniSphere 4.8.6
Post by: Rhuan on September 13, 2017, 08:58:24 pm
Adding in JSNumbertoInt and JSINttoNumber where possible resulted in somewhere between a 2% and 5% speed increase - probably worth doing but not the major issue.

All profiling points to JSCopyString being the most significant performance bottleneck unfortunately it seems like the majority of api calls rely on it heavily for one thing or another.
Title: Re: miniSphere 4.8.6
Post by: Rhuan on September 13, 2017, 10:41:21 pm
So I tried implementing this: https://github.com/Microsoft/ChakraCore/issues/3623

It was a surprisingly simple addition to the Chakra source, the below in JSRT.cpp + a stub in ChakraCore.h
Code: [Select]
//proposed new api to skip string type conversion
CHAKRA_API JSCreatePropertyIDfromJSString(
            _In_ JsValueRef stringValue,
            _Out_ JsPropertyIdRef *propertyId)
{
  const char16* str = nullptr;
  size_t strLength = 0;
  JsErrorCode errorCode = JsStringToPointer(stringValue, &str, &strLength);
  if (errorCode != JsNoError)
  {
    return errorCode;
  }
  return JsGetPropertyIdFromName(str, propertyId);
}

Then using that in make_property_id instead of the JSCopyString call; the result was a 20-25% performance improvement on common api calls; not enough to catch up with the old Duktape yet but it got rid of the majority of the slowdown in the Specs battle system - to the point that I would consider it playable.

I think we either should suggest this addition to ChakraCore OR we should just use it within the miniSPhere code base by using JSStringtoPointer - technically JSStringtoPointer is a windows only api BUT CC code uses it internally more than once so it is clearly functional on other platforms just as long as one is careful with sizes.

To optimise further I think it's worth tidying up uses of int and double; my brief testing earlier suggested that letting CC do the type conversion saves a few CPU cycles over doing type conversion in the miniSphere code.

I also think that all uses of JSCopyString should be examined, that function is just anti-speed.
Title: Re: miniSphere 4.8.6
Post by: Fat Cerberus on September 13, 2017, 10:46:19 pm
20-25% is nothing to sneeze at, maybe it's time for me to send another pull request upstream ;)
Title: Re: miniSphere 4.8.6
Post by: Fat Cerberus on September 13, 2017, 10:55:23 pm
Then using that in make_property_id instead of the JSCopyString call; the result was a 20-25% performance improvement on common api calls; not enough to catch up with the old Duktape yet but it got rid of the majority of the slowdown in the Specs battle system - to the point that I would consider it playable.

I think make_property_id() is only half of the puzzle - by removing the string copy from there, you've solved the problem for strings used as property keys (and 20-25% improvement indicates that happens a lot), but the problem for jsal_get_string() remains.  That comes into play whenever an API takes a string parameter, and is a much more difficult problem to solve.  Essentially the entire engine would need to be refactored to use UTF-16 internally.
Title: Re: miniSphere 4.8.6
Post by: Rhuan on September 13, 2017, 11:24:42 pm
Then using that in make_property_id instead of the JSCopyString call; the result was a 20-25% performance improvement on common api calls; not enough to catch up with the old Duktape yet but it got rid of the majority of the slowdown in the Specs battle system - to the point that I would consider it playable.

I think make_property_id() is only half of the puzzle - by removing the string copy from there, you've solved the problem for strings used as property keys (and 20-25% improvement indicates that happens a lot), but the problem for jsal_get_string() remains.  That comes into play whenever an API takes a string parameter, and is a much more difficult problem to solve.  Essentially the entire engine would need to be refactored to use UTF-16 internally.

To slightly improve speed could you use: JsGetStringLength(_In_ JsValueRef value, _Out_ int *length) to size the buffer rather than calling JsCopyString twice?

I think it will be tricky to take it much beyond that. The next slowest function looks like JsAddRef now - this is clearly much much slower than the Duktape equivalent.
Title: Re: miniSphere 4.8.6
Post by: Fat Cerberus on September 13, 2017, 11:35:02 pm
Hm, that might work.  JsGetStringLength() returns the length of the string in utf16 characters (the same as doing str.length in JS), in order to guarantee that will fit when transcoded to utf8 (1-4 bytes per character) one would need to allocate an oversized buffer at 4x the length.  Not very space-efficient, but I guess it would avoid doing a utf16 decode twice so it might improve performance.

My own results re: make_property_id concur with yours: Changing the function to use JsStringToPointer instead of JsCopyString took CPU usage in Specs from 16% with the console open down to ~12%, and the framerate drops almost completely went away.  So I think if we could avoid ever calling JsCopyString at all, we'd probably get pretty close the Duktape results for performance of API calls.

I don't understand why JsAddRef() is expensive though: In theory it should just be incrementing a reference counter which would be practically free.  I guess it'd be interesting to compile a debug build of ChakraCore and step into that function to see what it's actually doing under the hood...
Title: Re: miniSphere 4.8.6
Post by: Fat Cerberus on September 14, 2017, 01:56:21 am
I figured out why VertexList ctor got slower despite not dealing with strings at all: JSAL handled array indices by doing sprintf("%d", index) before calling jsal_get_prop.  That's unnecessary overhead since CC has JsGetIndexedProperty for exactly this purpose.  Growing pains, as always when learning a new library... :-[

It seems there's a lot of opportunity for optimization in JSAL itself, this is the price I pay for slapping it all together in a week while at the same time being a complete newcomer to Chakra's API :P
Title: Re: miniSphere 4.8.6
Post by: Rhuan on September 14, 2017, 02:53:59 pm
More to be done but signifcant progress being made, see attached latest results of the same speed tests for various ChakraCore miniSphere functions, whilst these are still slower than the duktape equivalents they're a lot better than they were and I believe Fat Cerberus is still working on another significant optimisation.

I would also like to highlight a different point, sure the sphere api functions are a currently a little slower in the CC version BUT the pure javascript execution is faster to a crazy degree; basically with the chakracore version of miniSphere the only measurable time taken to execute your game code with almost any test we've managed to give it is the api functions, any logic you use and any JS built ins are close enough to instant that measuring them is almost pointless (unless you have a loop run 1,000,000 times, though even then it won't exactly take long).

Who'd have thought that a change to string handling mechanisms would have been so significant. (Duktape's internals use UTF8 strings whereas ChakraCore's use UTF16 strings - a key aspect of miniSphere's API was built around sending messages back and forth as javascript strings this meant that with the CC implementation there are now lots of utf8->utf16->utf8 conversions which were the key cause of the slowdown)
Title: Re: miniSphere 4.8.6
Post by: Fat Cerberus on September 16, 2017, 12:19:38 am
Thanks to @Rhuan's help, we've managed to majorly optimize miniSphere's API layer; on my laptop (6th gen i7 quad), API call speed has improved fourfold.  See image attached.
Title: Re: miniSphere 4.8.7
Post by: Fat Cerberus on September 16, 2017, 02:24:44 am
4.8.7 fixes a bad bug where Sphere 1.x APIs won't accept a path beginning with ~/ (which unlike v2, is relative to the game manifest) and will instead throw "no save ID defined".
Title: Re: miniSphere 5.0b1 (stable: 4.8.7)
Post by: Fat Cerberus on September 16, 2017, 12:27:44 pm
The first beta version of miniSphere 5.0 is available now to allow people to test out the native ES6 support and vastly improved JavaScript performance! :D
Title: Re: miniSphere 5.0b1 (stable: 4.8.7)
Post by: Rhuan on September 16, 2017, 04:08:21 pm
Thanks to @Rhuan's help, we've managed to majorly optimize miniSphere's API layer; on my laptop (6th gen i7 quad), API call speed has improved fourfold.  See image attached.
In case anyone's worried those are not the times per call but rather the times to call each function 200,000 times (except for new VertexBuffer which is called only 20,000 times as it's so much slower than the others).
Title: Re: miniSphere 5.0b1 (stable: 4.8.7)
Post by: Rhuan on September 16, 2017, 04:56:02 pm
It's running smoothly on Mac, will try and have a neat version including only one copy of chakra as dylib (rather than 3) ready shortly. (miniSphere + SpheRun + cell all statically linked to Chakra = ~50mb which seems a little excessive)
Title: Re: miniSphere 5.0b1 (stable: 4.8.7)
Post by: Rhuan on September 16, 2017, 07:49:16 pm
miniSphere 5 beta for macOS Sierra
miniSphere 5 beta for macOS Sierra, including miniSphere, sphereRun, Cell and SSj:
edit: snip, now on downloads drive, see link in Fat Cerberus' post below

Be warned this is not fully tested in particular SSj with chakra has not been tested on macOS.
Also note the component apps will not work if moved into different folders to each other (this is because the ChakraCore dylib used by cell and sphereRun is hidden inside miniSphere.app )
Any comments or questions or other feedback please let me know.
Title: Re: miniSphere 5.0b1 (stable: 4.8.7)
Post by: Fat Cerberus on September 16, 2017, 11:52:22 pm
I put it on the Downloads drive: https://drive.google.com/drive/folders/0Bw-4UFVty4u1fjJ2RHNUTFNzMVp1WEFVOHlJaFk4STJPd1RMVmFLeGtDQWQ5SVlSSXhEWFk
Title: Re: miniSphere 5.0b1 (stable: 4.8.7)
Post by: Fat Cerberus on September 21, 2017, 01:53:46 am
Since ES6 modules will be natively supported in miniSphere 5.0, I'm going to deprecate require().  It will remain available for legacy compatibility with modules written for Node, but is otherwise no longer recommended for use in Sphere code.  import and export are much more elegant and will be the preferred mechanism for linking together scripts going forward.

Speaking of legacy compatibility, I recently added a new API, FS.evaluateScript(), which allows loading old-fashioned JS scripts.  I realized I was using RequireScript() a lot to load stuff like Aurora.js, which is awkward in an otherwise fully Sphere v2 codebase especially since all the v1 functions are technically deprecated... ;)
Title: Re: miniSphere 5.0b2 (stable: 4.8.8)
Post by: Fat Cerberus on September 21, 2017, 12:22:46 pm
miniSphere 5.0b2 and 4.8.8 are up.  Unlike the first beta, 5.0b2 has breaking API changes, so be sure to read the documentation to get up to speed.
Title: Re: miniSphere 5.0b2 (stable: 4.8.8)
Post by: Eggbertx on September 25, 2017, 03:56:09 pm
I'm not sure if this is a bug with ChakraCore or miniSphere, but I noticed an issue with miniSphere pointing to the wrong line number when an error is thrown
Code: (js) [Select]
// md = blah blah blah (unimportant)
md.bodyB = body; // line 47
md.target.Set(mouseX, mouseY); // line 48
But I'm getting this error:
Code: [Select]
ReferenceError: 'mouseX' is not defined
   at Anonymous function (src/main.mjs:47:4)
Title: Re: miniSphere 5.0b2 (stable: 4.8.8)
Post by: Fat Cerberus on September 25, 2017, 04:43:45 pm
The backtrace is generated by Chakra (it's taken from error.stack) so I guess it's either a bug, or else the line numbers are zero-based.
Title: Re: miniSphere 5.0b2 (stable: 4.8.8)
Post by: Fat Cerberus on September 26, 2017, 11:50:20 am
It's looking like I'm going to have to rewrite the module loader in order to support import(): the module loading API in Chakra is async, in other words they expect you to do module loading in the event loop rather than inline.  For static imports I could hack around that by just parsing all the dependent modules in a loop; for dynamic imports that's not possible because import() returns a promise and doesn't call your import callback until some time later.
Title: Re: miniSphere 5.0b2 (stable: 4.8.8)
Post by: East on September 28, 2017, 10:17:58 am
This might be a dumb question, but is there any way to force windowed mode to not start at 2x scale? It will help a lot with GIF recording.
Title: Re: miniSphere 5.0b2 (stable: 4.8.8)
Post by: Fat Cerberus on September 28, 2017, 10:27:03 am
It will help a lot with GIF recording.

Really weird use case for a game engine :P

The only way right now is to set the game's resolution high enough so it doesn't need to scale up.  I hardcoded the scaling for low resolutions because otherwise e.g. 320x240 is way too small at modern screen resolutions to play comfortably.
Title: Re: miniSphere 5.0b2 (stable: 4.8.8)
Post by: East on September 28, 2017, 10:39:56 am
My game is at 640x360, which I just found out is exactly on the edge for 2x.
640x361 works well for recording a gif to showoff new menu. :))
Title: Re: miniSphere 5.0b2 (stable: 4.8.8)
Post by: Rukiri on September 28, 2017, 11:30:16 am
My game is at 640x360, which I just found out is exactly on the edge for 2x.
640x361 works well for recording a gif to showoff new menu. :))
Try 960x540, it's a pretty standard base for resolution :)
Title: Re: miniSphere 5.0b2 (stable: 4.8.8)
Post by: Fat Cerberus on September 30, 2017, 01:45:58 am
Beta 3 is on the way, just one glitch with import() I want to work out first.  I've made even more optimizations since Beta 2 and native API call performance now surpasses that of mS 4.8.  Lots of other internal optimizations too, the engine should be faster and less CPU-hungry overall.
Title: Re: miniSphere 5.0b2 (stable: 4.8.8)
Post by: Fat Cerberus on October 01, 2017, 01:58:25 am
miniSphere 5.0b3 has just gone up, with the following enhancements over Beta 2:


I also changed the Random module back to the way it was prior to 5.0b2.  I had made it into a subclass of RNG but that made it excessively complex considering it was meant as a convenience module for when you don't actually need all the power of RNG.
Title: Re: miniSphere 5.0b3 (stable: 4.8.8)
Post by: Fat Cerberus on October 02, 2017, 08:41:27 am
I'm starting to put the finishing touches on miniSphere 5.0.  I'm not sure yet if there will be a 5.0b4 or if I'll go right to release-candidate stage for the next update.  I guess it depends how many bugs are reported in the meantime!

There is that crash on shutdown that @Rhuan keeps hitting, haven't been able to reproduce it yet... :(
Title: Re: miniSphere 5.0b3 (stable: 4.8.8)
Post by: Fat Cerberus on October 04, 2017, 03:51:31 am
Sphere.exit() turned out to be problematic as it was implemented using a longjmp and apparently Allegro didn't like that, causing a crash on exit.  So I replaced it with Sphere.shutDown() which simply instructs the event loop to exit, similarly to how ExitMapEngine() works in Sphere 1.x.

I also brought back the Pact class!  With async/await and the event loop taking center stage in miniSphere 5.0, I figured there should be an easy way in the API to fulfill or reject promises out-of-band.  With Pact, as long as you hold a reference to both the promise and the pact it was made from, you can settle the promise at any time.  It's very intuitive and doesn't require understanding closures.
Title: Re: miniSphere 5.0b3 (stable: 4.8.8)
Post by: DaVince on October 04, 2017, 07:33:51 am
Quote
Sphere.exit() turned out to be problematic as it was implemented using a longjmp and apparently Allegro didn't like that, causing a crash on exit.  So I replaced it with Sphere.shutDown() which simply instructs the event loop to exit, similarly to how ExitMapEngine() works in Sphere 1.x.
I understand changing the code to make it function, but why did you also change the method name? Sphere.exit() is nice and obvious.

I also gotta say, by the way - nice work on all of this! :D
Title: Re: miniSphere 5.0b3 (stable: 4.8.8)
Post by: Fat Cerberus on October 04, 2017, 09:09:47 am
"exit" implies that the method is immediate (which it was), "shutdown" tells the reader that there's a process to it all.  Just as you can either press the power button to forcibly turn off the computer or click "shut down" which will cleanly shut down the OS but has rules to it (open processes must close first, busy apps can block it, etc.)  Subtle difference, but very important from an API design standpoint.  It's the same reason I renamed FileStream#size to fileSize: I want the API to be as intuitive and self-describing as possible to a person reading the code as it will be to the person writing it.
Title: Re: miniSphere 5.0b3 (stable: 4.8.8)
Post by: DaVince on October 04, 2017, 09:35:04 am
Well, I asked exactly because exit() is very self-explanatory as well, and just something I've seen around more in software I guess. But your explanation makes a lot of sense; I'm just going to have to remember the change if I ever use the function. :P
Title: Re: miniSphere 5.0b3 (stable: 4.8.8)
Post by: Fat Cerberus on October 04, 2017, 09:52:37 am
API design is a very subtle art and it's been my experience that method naming isn't always just about the immediate effect of a method.  Consider the difference between:
Code: (javascript) [Select]
Sphere.exit();
Dispatch.onUpdate(doStuff);

Versus:
Code: (javascript) [Select]
Sphere.shutDown()
Dispatch.onUpdate(doStuff);

One might reasonably expect the former to close the engine regardless of what comes after.  It wouldn't, because the new job added to Dispatch keeps the event loop alive.  "But I explicitly said to exit, what's going on?!"  Whereas, if one looks at the latter code, it's requested a shutdown, but then added a new process afterwards which effectively cancels the request (you can't shut down if there are active processes).  It makes more sense intuitively that a request to "shut down" would be canceled (since an OS shutdown can do this too), canceling an "exit" (which sounds atomic) seems nonsensical.

Consider the documentation for ExitMapEngine(), where the note laying out the caveat is significantly longer than the description of the API:
https://github.com/sphere-group/sphere/blob/master/sphere/docs/development/api.txt#L495-L499

And that note is *STILL* vague--what exactly causes MapEngine to return if not the exit call itself?  I know the answer to that because I'm familiar with the Sphere 1.x source code and know that it's single-threaded.  But to the uninitiated it's not nearly as obvious.  For example if you start a FlipScreen loop in an update script that will block the exit despite performing other event loop-y operations.  Which is why the miniSphere documentation is much more explicit about it:
https://github.com/fatcerberus/minisphere/blob/master/docs/sphere2-core-api.txt#L236-L240

Control MUST return to the engine (i.e. ongoing JavaScript operations must run to completion) for the request to be honored.
Title: Re: miniSphere 5.0b3 (stable: 4.8.8)
Post by: Fat Cerberus on October 07, 2017, 11:04:39 am
miniSphere 5 beta 4 is on its way, it's just that I keep hitting really bad bugs that I have to fix.  Also I'm a refactoring fiend so a good deal of the regressions are my own fault, but let's not talk about that... ;)
Title: Re: miniSphere 5.0b3 (stable: 4.8.8)
Post by: Fat Cerberus on October 09, 2017, 01:44:28 am
5.0b4 is out now! 8)

The entire Sphere Runtime has been completely overhauled and brought some breaking changes with it (although I tried to be as conservative as possible), see the API documentation to find out what you'll have to do to upgrade.  Notably, Thread.join() and Scene#run() are no longer blocking and instead return a promise that you can await.

Major changes in Beta 4 off the top of my head:

Title: Re: miniSphere 5.0b4 (stable: 4.8.8)
Post by: Fat Cerberus on October 09, 2017, 09:57:41 am
Forgot to mention - because files were renamed, I strongly recommend uninstalling any previous version of miniSphere before installing 5.0b4 to ensure cruft from previous versions doesn't cause problems.
Title: Re: miniSphere 5.0b4 (stable: 4.8.8)
Post by: Eggbertx on October 17, 2017, 11:57:17 am
Do you have an ETA on a stable 5.0 release? I'm not complaining, I just haven't been updating the AUR package. I'm pretty sure I'm the only person who actually uses it, but I don't want to update to a potentially unstable release, and I don't feel like creating a second AUR package for unstable builds.
Title: Re: miniSphere 5.0b4 (stable: 4.8.8)
Post by: Fat Cerberus on October 17, 2017, 12:20:18 pm
I'm hoping to have the release ready by Halloween, but I'm not sure how feasible that is just yet.  There are a few showstoppers that may or may not be easy to fix:
https://github.com/fatcerberus/minisphere/issues?q=is%3Aissue+is%3Aopen+label%3Ashowstopper

So we'll see what happens.
Title: Re: miniSphere 5.0b4 (stable: 4.8.8)
Post by: Eggbertx on October 17, 2017, 05:50:50 pm
Alright, that's fine. Arch is known for being "bleeding edge", but I don't want to push something that isn't stable yet.
Title: Re: miniSphere 5.0rc
Post by: Fat Cerberus on October 23, 2017, 03:04:05 am
miniSphere 5.0rc is up.  This is a release candidate and should therefore be very stable.  As such, the links for miniSphere 4.8 have been taken down.  Give miniSphere 5.0 a spin, it's awesome!
Title: Re: miniSphere 5.0rc
Post by: Eggbertx on October 23, 2017, 10:33:24 am
Do you think it's safe enough for the AUR?
Title: Re: miniSphere 5.0rc
Post by: Fat Cerberus on October 23, 2017, 11:08:09 am
Release candidate is just that: Either it or a build very close to it will become the final release (unless someone finds some huge showstopping bug between now and the 31st).  So yeah, I definitely consider it stable at this point.
Title: Re: miniSphere 5.0rc
Post by: Fat Cerberus on October 27, 2017, 04:23:08 pm
I made some last-minute tweaks to 5.0.0 to improve the experience compared to the RC, mostly to do with error reporting.  In particular was an issue where syntax errors in a module didn't have file:line info that made quick debugging much harder than it needed to be.  So that'll be fixed in the final build.

I also made a change to the API, shader uniforms are now set directly on the Shader instead of being associated with Models.  That matches how OpenGL shaders actually work and allowed me to optimize things better.

Release is still on track for the 31st so get ready, the best miniSphere release yet is almost here!  8)
Title: Re: miniSphere 5.0rc
Post by: Radnen on October 27, 2017, 09:53:41 pm
Release is still on track for the 31st so get ready, the best miniSphere release yet is almost here!  8)

Nice! Can't wait. :)
Title: Re: miniSphere 5.0rc
Post by: Fat Cerberus on October 28, 2017, 01:22:21 am
Here's a new time-saving feature coming to SSj 5.0: quick refs.  These give you a way to quickly drill down into objects using the examine command without having to keep retyping property names over and over again.  For example:

Code: [Select]
src/SEngine.mjs:230 [anonymous function]
(ssj) v
var  j        0
var  done     false
var  entity   {...} *42
var  vec      {...} *56
var  target   {...} *70
var  runTime  {...} *84
var  inputs   {...} *98

src/SEngine.mjs:230 [anonymous function]
(ssj) x *84
prop  __proto__    {...} *115
prop  HUD          {...} *116
prop  windowStyle  {...} *117
prop  input        {...} *118
prop  font         {...} *119
prop  x            5
prop  y            320
prop  width        630
prop  height       150
prop  queue        {...} *124
prop  keys         {...} *125

Speaking of time-savers, it's also no longer necessary to quote an expression for eval or examine when it contains spaces, you can just type, e.g., eval 2 + 2 and it will work. :)
Title: Re: miniSphere 5.0rc
Post by: DaVince on October 28, 2017, 02:35:40 pm
I've got more free time now and have started diving into miniSphere. I gotta say, the way you set things up it's actually pretty easy to get started, especially with all the useful core libraries and the debugger! Nice work. :smiley_cat:
Title: Re: miniSphere 5.0.0
Post by: Fat Cerberus on October 31, 2017, 12:28:58 am
Just in time for Halloween :jack_o_lantern:, miniSphere 5.0 final has been released, bringing a scary amount of improvements including long-awaited support for ES2015+ and a blazing-fast JavaScript engine in ChakraCore.  miniSphere 5.0 also brings some brand-new Sphere Runtime modules, many API refinements and freezes the Core API to ensure that code written for Sphere v2 today will continue to work in all future engine versions.*  SSj has also been updated with some time-saving features.

This release includes many, many breaking changes over miniSphere 4.8.x, so be sure to look over the release notes and changelog here:
https://github.com/fatcerberus/minisphere/releases

Be sure to uninstall any previous version of miniSphere, including pre-release versions of 5.0, before setting up 5.0.0.  Many files were removed or renamed, so it's good to start with a clean slate to ensure cruft from previous builds doesn't interfere.

* Note: API freeze applies only to the Core API in miniSphere 5.0; the Sphere Runtime API freeze is currently scheduled for the next major version release (6.0).
Title: Re: miniSphere 5.0.0
Post by: DaVince on October 31, 2017, 10:51:40 am
Nice work, and a solid release! :))
Title: Re: miniSphere 5.0.0
Post by: Fat Cerberus on October 31, 2017, 03:30:50 pm
There's now a macOS build of 5.0.0 available for download courtesy of @Rhuan.  With that, miniSphere now officially supports all three major desktop platforms.  Hooray! :smile:
Title: Re: miniSphere 5.0.0
Post by: Radnen on October 31, 2017, 05:27:30 pm
Great work!! I'll definitely check this out when I get home!
Title: Re: miniSphere 5.0.0
Post by: Eggbertx on October 31, 2017, 09:15:31 pm
The AUR package has been updated to the latest upsteam version. And the main Allegro package finally works nicely, so minisphere's AUR package doesn't have to depend on an unofficial package.
Title: Re: miniSphere 5.0.0
Post by: Fat Cerberus on October 31, 2017, 09:25:21 pm
The AUR package has been updated to the latest upsteam version.

Thanks, I guess you had no problem getting ChakraCore running on Arch then.
Title: Re: miniSphere 5.0.0
Post by: Eggbertx on October 31, 2017, 10:14:00 pm
Nope, no issues.
Title: Re: miniSphere 5.0.1
Post by: Fat Cerberus on November 02, 2017, 02:20:20 am
Released 5.0.1 to fix a handful of small but potentially nasty bugs. :beetle:
https://github.com/fatcerberus/minisphere/releases/tag/v5.0.1
Title: Re: miniSphere 5.0.1
Post by: Eggbertx on November 02, 2017, 09:25:32 am
I ran into the Exit() bug the other day, so I'm glad to see that's fixed.
Title: Re: miniSphere 5.0.1
Post by: Eggbertx on November 17, 2017, 10:39:11 pm
I'm not sure if it's something I'm doing wrong, but calling Sphere.restart() seems to cause a massive slowdown
Title: Re: miniSphere 5.0.1
Post by: Fat Cerberus on November 18, 2017, 01:56:29 am
I'm not sure if it's something I'm doing wrong, but calling Sphere.restart() seems to cause a massive slowdown

That's a weird one.  I'll definitely look into that!

It could be that closing down CC and reinitializing it disables the JIT somehow, but that level of slowdown shouldn't be noticeable unless you're doing something really intense like mp3 decoding (Chakra's bytecode interpreter is faster than Duktape's).  So it's probably something else.
Title: Re: miniSphere 5.0.1
Post by: Fat Cerberus on November 19, 2017, 01:57:06 pm
@Eggbertx Okay, I can reproduce the bug.  Even Specs (which worked fine under Duktape) drops to like 15fps after calling Sphere.restart().  So I don't think it's related to CC at all.  I'll profile the engine to investigate further where the slowdown might be.
Title: Re: miniSphere 5.0.1
Post by: Eggbertx on November 19, 2017, 03:53:59 pm
Alright, that's good (for me at least). I wasn't sure if it was something I was doing wrong, because I'm rewriting my old robotfindskitten port, and I was worried that it might be bogging it down.
Title: Re: miniSphere 5.0.1
Post by: scojin on November 19, 2017, 05:19:49 pm
I recently updated from 4.5 to 5.0.1 so I already know I'm going to run into a heap of issues with the project I'm working on. I got the cellscript working and have a successful build, but I'm getting errors in my main script when importing my own modules.
I'm sure there is a lot of nuance and order that is over my head, so is there an example demo that I could view? I've looked at the getting started tutorial and it helped a little bit, but I've got a small rat's nest that's gotta be sorted through.

Any tips on updating projects from the older version of minisphere? Any recommended reading would be great, too!
Thanks!
Title: Re: miniSphere 5.0.1
Post by: Rhuan on November 20, 2017, 12:59:59 pm
I recently updated from 4.5 to 5.0.1 so I already know I'm going to run into a heap of issues with the project I'm working on. I got the cellscript working and have a successful build, but I'm getting errors in my main script when importing my own modules.
I'm sure there is a lot of nuance and order that is over my head, so is there an example demo that I could view? I've looked at the getting started tutorial and it helped a little bit, but I've got a small rat's nest that's gotta be sorted through.

Any tips on updating projects from the older version of minisphere? Any recommended reading would be great, too!
Thanks!
This should probably be a topic in support rather than engine dev but :P

The getting started tutorial is horribly out of date right now... :(

One important change if moving from 4.x to 5.x is to get rid of uses of transpile() in your cell script - scripts should just be install()'ed with 5.x.

Also if using the import operator scripts need the extension ".mjs" rather than ".js".

Here's a link to a simple example project in miniSphere 5.0, very early demo of my remake of Kefka's Revenue (mostly just showing off the map engine I've made):
https://www.dropbox.com/s/x5sfkcad3jjhk91/KRdemo.zip?dl=0

If you need more specific help saying what error messages you're getting exactly and posting some of your code would make it easier.

If the problems are just with import then maybe read the MDN article on it: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Statements/import
The common places I expect people to go wrong with it are:
a) get the file path wrong (import expects relative paths)
b) confuse default exports with other named exports
c) try and use static import syntax in a dynamic way
d) not use the .mjs extension that miniSphere requires
Title: Re: miniSphere 5.0.1
Post by: scojin on November 21, 2017, 05:26:36 pm
Okay, thanks for the help.
Issues were file extensions(.js to .mjs) and getting rid of the tutorial changes.
I'll start a new thread if I have anymore questions.
Title: Re: miniSphere 5.0.1
Post by: Fat Cerberus on November 21, 2017, 11:39:51 pm
a) get the file path wrong (import expects relative paths)

Not entirely accurate - you can use a full path so long as it has a SphereFS prefix, e.g. @/bin/awesome.mjs.
Title: Re: miniSphere 5.0.1
Post by: Eggbertx on December 13, 2017, 09:22:57 am
I've been busy with IRL things so I haven't gotten a chance to work on it yet, but I was thinking about how I'm going to implement a mobile specific API, and I wanted to get your opinion on it before I start. Hopefully it will be compatible with both Android and iOS, but I don't have a macOS computer and don't feel like jumping through hoops.

Code: [Select]
`MobileDevice` Object
----------------------

MobileDevice.Default [read-only]

 Gets the device running miniSphere. This will likely only exist on mobile builds

MobileDevice.Platform [read-only]

 Can be 'android' or 'ios'

MobileDevice#activateTiltSensor()

 Activates the device's tilt sensor. If it is not activated, calling
 MobileDevice#getTilt() will return null

MobileDevice#deactivateTiltSensor()

 Deactivates the device's tilt sensor

MobileDevice#getTilt()

 Returns an object with x, y, and z float values representing the device's angle

MobileDevice#activateAccelerometer()

 Activates the device's accelerometer. If it is not activated, calling
 MobileDevice#getAcceleration() will return null

MobileDevice#deactivateAccelerometer()

 Deactivates the device's accelerometer

MobileDevice#getAcceleration()

 Returns an object with x, y, and z float values representing the device's
 current acceleration

MobileDevice#deactivateAllSensors()

 Deactivates all currently activated sensors

MobileDevice#getTouchCoordinates()

 Returns an array of objects with x and y coordinates for the current number
 of touches. If the screen isn't being touched, it will return a 0-length array.

Thoughts?
Title: Re: miniSphere 5.0.1
Post by: DaVince on December 13, 2017, 04:22:20 pm
Looks pretty good to me, and generic enough to Just Work on both platforms.
Title: Re: miniSphere 5.0.1
Post by: Eggbertx on December 13, 2017, 08:40:30 pm
That was the idea. Also, I may or may not have brought this up before, but most games developed for desktop miniSphere won't be compatible with mobile miniSphere and vice versa without some heavy modification.
Title: Re: miniSphere 5.0.1
Post by: Fat Cerberus on December 24, 2017, 02:18:23 pm
I think the engine needs moar eaty pigz
Title: Re: miniSphere 5.1.0
Post by: Fat Cerberus on December 25, 2017, 02:22:04 am
miniSphere 5.1.0 is out just in time for Christmas!  Not as many new features in this release as I originally planned, but I wanted to get a release out as it's been almost two months since 5.0.1. and I've done several important bug fixes since then.

Sphere API level is now 2.

Merry Christmas! :christmas_tree:
Title: Re: miniSphere 5.1.0
Post by: Radnen on December 25, 2017, 01:18:56 pm
Merry Christmas! What you've done with Sphere in the last year is truly astonishing! Great work!!
Title: Re: miniSphere 5.1.0
Post by: Radnen on December 30, 2017, 11:09:37 pm
So, I download a fresh, clean copy of MiniSphere 5.1.0 and I run a new, default game in the editor and I get a reference error. It says 'screen' is not defined.

I'm thinking of maybe creating games again, hahaha. I have a Raspberry Pi and I'd love to see if maybe it's possible to have Sphere games run on it someday.
Title: Re: miniSphere 5.1.0
Post by: Fat Cerberus on December 30, 2017, 11:11:59 pm
That's odd, because 5.1.0 was supposed to fix that:
https://github.com/fatcerberus/minisphere/commit/51de201d53df8e4dc82afc6cff9f969f0a298890

So yeah, change any instances of "screen" to "Surface.Screen".
Title: Re: miniSphere 5.1.0
Post by: Radnen on December 30, 2017, 11:20:01 pm
I actually just fixed that with Surface.Screen. I installed the engine and editor with the installer and I indeed see 5.1.0 in the about config.

Anyways, I'm going to try and see what I can do with the new API. Things are much different. Better, but different so, I'll have to learn a few things.
Title: Re: miniSphere 5.1.0
Post by: Fat Cerberus on December 30, 2017, 11:21:56 pm
Protip: The Sphere 1.x API is still available, and for the most part can be freely mixed with v2 API code.
Title: Re: miniSphere 5.1.0
Post by: DaVince on December 31, 2017, 09:32:59 pm
@Radnen If you feel like it, you could join the Discord chat while experimenting with the new API. Really helps to leave feedback on it especially when you find a bug, or you'll get a quick response on how something works. :)
Title: Re: miniSphere 5.1.0
Post by: Rhuan on January 20, 2018, 04:51:51 am
Bad news I'm afraid regarding the miniSphere build for macOs. It appears that a new security feature of some kind in macOs High Sierra breaks it. (The error message you'll get is "can't find system font")

The build should still run fine on macOs Sierra though.

For anyone who's at the bleeding edge with macOs I'll try and find a way around this but not really sure what the answer is yet - it may have t have an installer or the answer may be that I just to put the system folder inside the app bundle - and change the path handling to expect this.

For now sphereRun is still usable, so if you're ok with having to launch everything from the command line it's all good... (I assume apple doesn't apply security to unix style applications in the same way they do to app bundles)
Title: Re: miniSphere 5.1.1
Post by: Fat Cerberus on January 24, 2018, 12:07:32 am
miniSphere 5.1.1 is up and fixes a bug where the engine could crash on startup if a game had any circular import dependencies.  The macOS build has been updated to the latest version as well.  Special thanks to @Rhuan for this release.
Title: Re: miniSphere 5.1.1
Post by: Fat Cerberus on January 31, 2018, 01:39:39 am
@Rhuan and I have been working on a great new debugging aid which is coming in miniSphere 5.2.  Historically, rejecting a promise in miniSphere would crash the specific promise chain it occurs in, but otherwise go unnoticed.  This can cause some very strange and difficult-to-understand behavior in heavily asynch code, especially in codebases where async functions are used heavily (like my own Spectacles project).  To illustrate the problem, take this code:

Code: (JavaScript) [Select]
async function pig()
{
    SSj.log("Oh no, I think I'm going to get eaten by the pig...");
    throw new Error("PIG!!!");
    SSj.log("The pig totally ate me, you guys!");
}

pig();
SSj.log("The pig is coming to eat EVERYTHING.");

This will hit all the SSj.log() calls except for the second one in pig(), and terminate normally.  That happens because pig(), being an unawaited async call, crashes at the throw without affecting the primary (i.e. synch) code path; the promise representing its return value is simply rejected sight unseen.

Broken promises are annoying and difficult to debug, but thanks to @Rhuan's hard work getting promise rejection tracking into ChakraCore, in miniSphere 5.2, the code above will be detected as a runtime error (by the event loop) and generate an error screen with a stack trace the same way an exception in synch code does.  This will make debugging async code much more pleasant.
Title: Re: miniSphere 5.1.1
Post by: Fat Cerberus on February 02, 2018, 01:37:17 am
Heads up: The next Windows release will be 64-bit only.  I've considered doing this a few times before, but kept backpedaling since I still wanted to support XP/Vista.  There are few factors that make going full 64-bit make more sense now:


So yes, long story short: miniSphere 5.2 5.1.2 and later will require a 64-bit operating system on all platforms.

edit: This change came earlier than expected and has been implemented starting with miniSphere 5.1.2.
Title: Re: miniSphere 5.1.2
Post by: Fat Cerberus on February 07, 2018, 12:57:46 pm
miniSphere 5.1.2 improves exception handling for asynchronous code.  Unhandled rejected promises--and by extension, uncaught exceptions in async functions--will now trigger the error screen the same as a normal exception.  It also adds system-directory packaging: When packaging a game into an SPK, Cell will now include the system files to future-proof the package (do note that the packaged system files will be ignored unless the package is run with v5.1.2 or later).

These are somewhat large additions for a point release (and technically violate semantic versioning rules), but they are low-risk changes and I wanted to get these out sooner rather than later, particularly the async exception handling.

Note: In line with the post above, the Windows release of miniSphere 5.1.2 is 64-bit only.
Title: Re: miniSphere 5.1.2
Post by: Eggbertx on February 13, 2018, 09:31:24 am
Removing 32-bit support seems pretty reasonable, considering Windows 7 was released 8 years ago according to Wikipedia. And that's a long time in technology years.
Title: Re: miniSphere 5.1.3
Post by: Fat Cerberus on February 19, 2018, 12:21:58 am
5.1.3 is up to fix a couple annoying issues and add support for Promise#finally().  The error screen is also a bit more colorful now. :smile:
Title: Re: miniSphere 5.1.3
Post by: Rhuan on February 26, 2018, 06:22:40 pm
MacOS release of 5.1.3 link here until @Fat Cerberus uploads a copy: https://www.dropbox.com/s/tbv7fohz60iigu9/miniSphere%205.1.3.zip?dl=0
Title: Re: miniSphere 5.1.3
Post by: Fat Cerberus on March 03, 2018, 02:01:06 pm
I've updated the link for the macOS build in the OP to 5.1.3.  All supported platforms are now up to date. :D
Title: Re: miniSphere 5.1.3
Post by: Fat Cerberus on March 21, 2018, 02:43:11 am
I've made some improvements under the hood that should make miniSphere 5.2 noticibly snappier than 5.1 in various scenarios.

Also some new features coming in the next release, stay tuned for more info. :D
Title: Re: miniSphere 5.1.3
Post by: Fat Cerberus on March 25, 2018, 01:43:29 pm
Regarding the optimizations mentioned above, performance of API calls in particular has sped up a ton compared to the current release (5.1.3).  Before-and-after image attached.  This is a huge improvement and the effects are very noticeable in-game as well; everything is much snappier and CPU usage has been nearly cut in half across the board.
Title: Re: miniSphere 5.2b1 (stable 5.1.3)
Post by: Fat Cerberus on April 07, 2018, 02:20:32 am
The first beta of miniSphere 5.2 is here!  I don't normally do beta releases for minor versions, but this one turned out to be bigger than I anticipated...

Big new features showed off in the beta are:

Title: Re: miniSphere 5.2b1 (stable 5.1.3)
Post by: Radnen on April 08, 2018, 10:45:59 pm
  • Dispatch.onExit: Run some code on shutdown, after the event loop terminates.  This works even if the user manually closes the window!

I like this. This prevented me from writing a clean MMO in Sphere back in the day. I had to 'poll' every user to see if they were actually logged in or not every like 5 minutes. Good times, hahaha,
Title: Re: miniSphere 5.2b1 (stable 5.1.3)
Post by: Fat Cerberus on April 09, 2018, 12:13:16 am
It was actually somewhat tricky to implement that, as miniSphere is fully v1 compatible and Sphere 1.x didn't have an event loop.  So if you close the window while the game is in the middle of while (true) { FlipScreen(); }, the only way the engine can honor the exit request at that point is to forcibly bail out of JavaScript execution.  With ChakraCore that's done by calling a function to disable script execution, and doing so works well enough... the tricky part is that if you leave it disabled, you'll fall right through the event loop without getting a chance to run any exit handlers.  Figuring out where to put the safety net to stop that from happening without causing other unwanted side-effects was... fun. :stuck_out_tongue:
Title: Re: miniSphere 5.2b2 (stable 5.1.3)
Post by: Fat Cerberus on April 15, 2018, 12:56:03 am
miniSphere 5.2 beta 2 is now available with more bug fixes and optimizations, better Sphere v1/v2 interop (including tweaks to how FlipScreen is handled in v2 code), support for all Sphere v1 surface blend modes and the ability to use Z.deflate and Z.inflate in Cell.
Title: Re: miniSphere 5.2b2 (stable 5.1.3)
Post by: Fat Cerberus on April 15, 2018, 02:44:56 pm
Found a bug in beta 2: Calling MapEngine() or FlipScreen() from the event loop will cause the engine to crash on exit. :disappointed:

I have an idea of the cause already; fix will be in either beta 3 or 5.2.0 final, whichever comes first.
Title: Re: miniSphere 5.2b3 (stable 5.1.3)
Post by: Fat Cerberus on April 22, 2018, 02:25:08 am
This just in!  miniSphere 5.2b3 is up and fixes more bugs and brings new features for backward compatibility testing: 1) An apiLevel field in the game manifest for targeting a specific version of the API, and 2) --retro command-line option for SpheRun that makes the engine emulate the targeted API level (anything newer is disabled).
Title: Re: miniSphere 5.2b3 (stable 5.1.3)
Post by: Eggbertx on April 26, 2018, 07:16:44 pm
Has there been any update on the possibility of TTF support? We wouldn't necessarily need advanced features to be exposed to the JavaScript API starting out, but basic support would be nice.
Title: Re: miniSphere 5.2.0
Post by: Fat Cerberus on April 27, 2018, 06:50:59 pm
I agree TTF support would be nice to have, raster fonts work well enough in most cases but there's nothing like having an infinitely scalable typeface.  I'd want to give the API some thought first, as even if we just start with the bare essentials it needs to be extensible enough to remain stable.

I'll think about the best way to proceed on this front.
Title: Re: miniSphere 5.2.1
Post by: Fat Cerberus on April 29, 2018, 01:30:27 am
I forgot to mention before for anyone who hasn't already noticed, miniSphere 5.2 is now out of beta and brings a ton of new stuff: profiling, retrograde mode (old API emulation), blendmodes for Sphere v2, Dispatch.onExit() and much more.  Also, a lot of overhauling was done under-the-hood to give a very nice performance boost compared to v5.1. :smile:
Title: Re: miniSphere 5.2.2
Post by: Fat Cerberus on May 01, 2018, 01:01:31 am
miniSphere 5.2.2 is now available to fix a nasty bug in 5.2.0 and 5.2.1 that prevented double-clicking on minisphere.exe to launch the engine.  I'm a bit ashamed that this slipped under the radar, there's really no excuse for me not picking something that obvious up before the release.

Oh well, in any case it's fixed now. :sweat_smile:
Title: Re: miniSphere 5.2.3
Post by: Fat Cerberus on May 04, 2018, 12:49:09 pm
Because miniSphere 5.1 shipped with Sphere.APILevel set to 2, in the interest of maintaining backward compatibility the APIs introduced in that release have been retroactively frozen and the current experimental API level increased to 3.

APIs added in level 2:

edit: The freeze mentioned above was reversed thanks to the release of 5.0.2 and 5.1.4.  Disregard this post.
Title: Re: miniSphere 5.2.4
Post by: Fat Cerberus on May 11, 2018, 02:14:18 am
miniSphere 5.2.4, 5.1.4 and 5.0.2 are up.  I did a simultaneous release across three minor versions to bring the API back into parity, since backward compatibility had drifted due to important bug fixes (for example 5.0 didn't support index.mjs) and the recent addition of system-directory packaging.  Now if you target API level 1 and the game works properly with --retro it is also guaranteed to work in miniSphere 5.0 (which notably is the last 32-bit version), as it should be.

No functional changes in 5.2.4 other than to drop the reported API level back down to 1.  Since I updated 5.1 to realign the level 1 API, the premature freeze mentioned above is no longer necessary.

5.0.2 and 5.1.4 are available on either the miniSphere GitHub Releases page or the Downloads Drive, if anyone needs them.
Title: Re: miniSphere 5.2.5
Post by: Fat Cerberus on May 12, 2018, 02:40:32 am
5.2.5 is up as a hotfix for a regression that prevented miniSphere 5.2.4 from being able to run standalone .js and .mjs scripts (it would show an unsupported API level error and quit).  Sorry about that!
Title: Re: miniSphere 5.2.7
Post by: Rukiri on May 24, 2018, 09:42:28 am
Was image replaced with something else? It was completely remove from the sphere-runetime and game-modules, however Sphere Studio will still call the default project which still uses the image module.

Or am I missing something?
Title: Re: miniSphere 5.2.7
Post by: Fat Cerberus on May 24, 2018, 11:25:37 am
Yeah, the default project has been broken since miniSphere 5.0.0, that release changed the API significantly compared to 4.8 but it seems that I neglected to test creating a new project before I did the release.  I'll fix this in 5.2.8.  There was also a separate, more serious issue I discovered where several APIs are missing due to a bug in apiLevel handling.

Note that API Level 1 is frozen since 5.0.0 (the system modules are not, but I do try not to break them unnecessarily), so breakage like this should be minimized going forward.
Title: Re: miniSphere 5.2.9
Post by: Fat Cerberus on June 05, 2018, 01:05:13 am
miniSphere 5.2.9 is up.  It fixes a few minor SSj bugs as well as a bigger issue where certain combinations of circular module dependencies and import() (or require()) can crash the engine.  Barring major unforeseen issues (read: a showstopping bug), this will be the final release in the 5.2 series so that work can commence on miniSphere 5.3.
Title: Re: miniSphere 5.2.9
Post by: Fat Cerberus on June 07, 2018, 11:01:40 pm
Forgot to mention this above, the module fix besides fixing the aforementioned crash, also corrects module load order which was previously reversed, such that imports will now correctly execute in the expected order, e.g. this will now work properly:
Code: [Select]
import 'side-effect-module-1';
import 'side-effect-module-2';

In miniSphere versions prior to 5.2.9, such imports would execute in reverse order, which could cause issues especially in cases where modules are pulled in only for their side effects, à la legacy RequireScript. Special thanks to @Rhuan for fixing this in ChakraCore.
Title: Re: miniSphere 5.2.11
Post by: Fat Cerberus on July 01, 2018, 01:15:13 am
I released miniSphere 5.2.11 which fixes a bug with dynamic import() and updates ChakraCore to the latest stable release (1.10.0).  This is the final version of miniSphere 5.2 so that I can focus on coming up with ideas for 5.3.  As always though, if there are any issues let me know so they can get fixed. :smiley:
Title: Re: miniSphere 5.2.12
Post by: Fat Cerberus on August 18, 2018, 12:36:56 am
Okay, so I know I said that 5.2.11 would be the last 5.2 release, but the bug fixes were starting to pile up so I decided to put out another patch.  Besides bug fixes I also added an explanation of API levels to the documentation as well as showing which level each function requires.  Experimental functions are marked as such.
Title: Re: miniSphere 5.3 beta 1 (stable: 5.2.13)
Post by: Fat Cerberus on October 02, 2018, 01:09:01 pm
The first beta of miniSphere 5.3 is out, bringing some new APIs and several useful features, including cell init for initializing a new, working project directly from the command-line (no Sphere Studio required!).  More will be coming, but this seemed like a good point to push out a beta to hopefully get some field testing in on the new features, especially cell init.
Title: Re: miniSphere 5.3b1 (stable: 5.2.13)
Post by: Fat Cerberus on November 17, 2018, 03:04:07 pm
Starting in miniSphere 5.3, from() will be built into the Core API without the need to import the "from" module.  Like @Radnen 's Link.js library that came before it, from() queries are often incredibly useful in battle engines, and now I've written some code in the engine to compile these queries directly to JS for super-fast performance.

Code: (JavaScript) [Select]
let dinnerAmount = from(worldPopulation)
    .where(it => pig.isHungryFor(it))
    .besides(it => pig.devour(it))
    .reduce((a, it) => a + it.weight, 0);
SSj.log(`the pig just gained ${dinnerAmount} lbs.`);
Title: Re: miniSphere 5.3b1 (stable: 5.2.13)
Post by: Fat Cerberus on November 20, 2018, 01:10:51 am
So when I said above that from queries would be blazing fast, I meant it, and to prove it I've done a quick benchmark of other similar solutions:
Code: (javascript) [Select]
let fromQuery = new Query()
.where(it => it % 2 === 0)
.take(10000)
.select(it => it + 1)
.reduce((a, it) => a + it, 0);
Code: [Select]
 event                    count   time (us)   % run  avg (us)    % avg |
------------------------------------------------------------------------
 Array method chain       1,001   3,605,798  34.1 %     3,602   34.3 % |
 Underscore chain         1,001   2,065,016  19.5 %     2,062   19.6 % |
 Lodash chain             1,001   1,168,390  11.0 %     1,167   11.1 % |
 from.js 1.0              1,001     903,805   8.5 %       902    8.6 % |
 from.ts (Oozaru)         1,001     875,786   8.3 %       874    8.3 % |
 Link.js query            1,001     475,447   4.5 %       474    4.5 % |
 Link.js query (NR)       1,001     325,084   3.1 %       324    3.1 % |
 from.js 2.0              1,001     319,423   3.0 %       319    3.0 % |
 Lazy.js sequence         1,001     270,130   2.6 %       269    2.6 % |
 Sphere from() query      1,001     205,757   1.9 %       205    2.0 % |
 Sphere Query object      1,001     188,966   1.8 %       188    1.8 % |
 handwritten 'for' loop   1,001     119,092   1.1 %       118    1.1 % |

Sphere from() query and Sphere Query object is us!  All timings are for running a chain equivalent to the above query 1,000 times with the respective library over an array of 100,000 random integers between 0 and 1000.

Special thanks to @Radnen for (indirectly) giving me the idea - Link.js was touted as a replacement for writing repetitive for loops, which got me to thinking... what if I just compile the query to an actual for loop...

This is really incredible that I was able to get so close to native loop performance and is a big win for code readability.  Query chains remain understandable even with 10+ query operators chained together, but throw together a couple filters and mappings plus a sort (or two!) and the set of for loops you need to write to match it can get pretty gnarly.  Lodash is proof people are willing to sacrifice a great deal of performance to get more readable code (see benchmark results above), even in tight loops where it matters most, but it's even better if you don't have to. :smiley_cat:
Title: Re: miniSphere 5.3b1 (stable: 5.2.13)
Post by: Radnen on November 23, 2018, 02:56:01 pm
I'm still baffled the widespread popularity of lodash... when libraries you and I write can be an order of magnitude faster, and any missing convenience methods lodash has, can always be added.

Anyways, I didn't have from.js on me, but I ran your parameters on my new Linkier library along with lazy and Link and got:
Code: ('js') [Select]
const Linkier = require('./linkier.js');
const Link = require('./modules/link.js');
const Lazy = require('./modules/lazy.js');
const Benchmark = require('benchmark');

// Generate a random list of 100,000 integer between 0 and 1,000.
const items = [];
for (let i = 0; i < 100000; ++i) {
    items[i] = Math.floor(Math.random() * 1001);
}

const suite = new Benchmark.Suite;
const linkier = Linkier(items).filter(n => n % 2 === 0).take(10000).map(n => n + 1);
const link = Link(items).filter(n => n % 2 === 0).take(10000).map(n => n + 1);
const lazy = Lazy(items).filter(n => n % 2 === 0).take(10000).map(n => n + 1);

suite.add('Linkier', function() {
    linkier.reduce((a, n) => a + n, 0);
}).add('Link', function() {
    link.reduce((a, n) => a + n, 0);
}).add('Lazy', function() {
    lazy.reduce((a, n) => a + n, 0);
}).add('For Loop', function() {
    let n = 0;
    let r = 0;
    for (let i = 0; i < items.length; ++i) {
        if (items[i] % 2 === 0) {
            n++;
            if (n <= 10000) {
                const m = items[i] + 1;
                r = r + m;
            }
        }
    }
}).add('For Loop 2', function() {
    let n = 0;
    let r = 0;
    for (let i = 0; i < items.length; ++i) {
        if (items[i] % 2 === 0) {
            n++;
            if (n <= 10000) {
                const m = items[i] + 1;
                r = r + m;
            } else {
                break;
            }
        }
    }
}).on('cycle', (event) => {
    console.log(String(event.target));
}).on('complete', function() {
    console.log('Fastest is: ' + this.filter('fastest').map('name'));
}).run({ async: true });

Code: [Select]
Linkier x 4,242 ops/sec ±5.49% (78 runs sampled)
Link x 3,786 ops/sec ±2.13% (91 runs sampled)
Lazy x 2,917 ops/sec ±2.87% (91 runs sampled)
For Loop x 2,213 ops/sec ±2.01% (96 runs sampled)
For Loop 2 x 11,052 ops/sec ±1.52% (97 runs sampled)
Fastest is: For Loop 2

What we learn here, is that hand-writing a for loop can be slower... The first for loop doesn't short circuit after 10,000 items, but that's on intention. How many times in writing a for loop you think of doing that? Therefore from.js, link.js and other libraries are going to be faster since they can make those common-sense decisions for you, and just automatically make your code faster.
Title: Re: miniSphere 5.3b2 (stable: 5.2.13)
Post by: Fat Cerberus on November 30, 2018, 06:49:03 pm
miniSphere 5.3 release is very close - just a few more API kinks I have to work out but otherwise everything is ready to go.

I even made a last-minute API addition: you'll be able to create custom BlendOps to have better control over the blending stage of the graphics pipeline.  This is something that can't be handled in the fragment shader at all--blending is still fixed-function even on modern hardware.
Title: Re: miniSphere 5.3.0 RC1
Post by: Fat Cerberus on December 16, 2018, 02:08:54 am
miniSphere 5.3.0 RC1 is now available for download!  This is a release candidate, which means the code is essentially frozen; no further changes will be made other than bug fixes for 5.3.  If no major issues are reported, this exact build will become the final version of miniSphere 5.3.0.

There's way too much new stuff to list here (5.3 ended up way bigger than I originally planned), so just check out the release notes on GitHub:
https://github.com/fatcerberus/minisphere/releases/tag/v5.3rc1
Title: Re: miniSphere 5.3.0
Post by: Fat Cerberus on December 24, 2018, 11:57:03 pm
miniSphere 5.3.0 has just been released and is now available for download.  This version brings a ton of new APIs, a brand-new, faster version of from.js rewritten from the ground up, bumps the API level to 2, brings back support for 32-bit versions of Windows in the official release, adjusts script-loading semantics to allow using .js for modules (it's like a long lost friend coming back home :P), and so much more.

Be extra sure to check out the Release Notes (https://github.com/fatcerberus/minisphere/releases/tag/v5.3.0) for this release before getting started, as there are several potentially breaking changes due to modified semantics in a few places (the Core API remains backward compatible, as promised by the mS 5.0.0 API freeze).  And for Windows users, as always with milestone releases, uninstall your previous version of miniSphere before installing 5.3.0 to ensure all stale files get removed.

Merry Christmas everybody!
:santa: :christmas_tree::gift:
Title: Re: miniSphere 5.4.0
Post by: Fat Cerberus on December 01, 2019, 02:17:09 am
miniSphere 5.4.0 is out.  I kind of rushed the release because 5.3 was released almost a year ago; please let me know of any bugs!

This version adds support for the .cjs (CommonJS) extension and introduces a new development.strictImports flag to game.json which forces import to be Oozaru-compatible (i.e. it forces you to include the extension for any imports of scripts inside your project), and a bunch of other enhancements.

See the release page for the full list of changes:
https://github.com/fatcerberus/minisphere/releases/tag/v5.4.0
Title: Re: miniSphere 5.4.2
Post by: Fat Cerberus on February 20, 2020, 01:49:34 am
Just posted a quick update: miniSphere 5.4.2.  A few of the currently-experimental APIs were renamed, so be sure to check the changelog to see if the renames affect you.
Title: Re: miniSphere 5.5.0
Post by: Fat Cerberus on August 15, 2020, 09:21:17 am
I don't know if anyone still even uses the forums anymore, but...

Just posted a new miniSphere version, 5.5.0, with a bunch of miscellaneous changes.  Notably the .fromFile() APIs are canonized and API level has been increased to 3.

Just as a quick heads-up, there's an upcoming rebranding: While the engine itself will continue to be called miniSphere to differentiate it from Oozaru (the Web implementation), the download itself will be renamed to Sphere. Not sure when I'm going to do this, but it'll be soon.
Title: Re: miniSphere 5.5.0
Post by: Radnen on August 19, 2020, 03:19:40 pm
miniSphere is Sphere now. Good job Fat Cerberus! Congrats on the v5.5.0 release. :)
Title: Re: Sphere 5.5.1 Official Release (miniSphere)
Post by: Fat Cerberus on September 13, 2020, 01:44:35 pm
The Sphere 5.5.1 development kit has been released for Windows, beginning the rebranding effort mentioned a few posts back.
Title: Re: Sphere 5.5.1 (miniSphere)
Post by: Fat Cerberus on September 17, 2020, 12:26:51 pm
The next big feature I want to work on is Oozaru integration. Not sure how that's going to go yet, but the basic idea is to do something where Cell can include a copy of Oozaru with the compiled project. Then you just need to find a place to host it and you're good to go.
Title: Re: Sphere 5.5.1 (miniSphere)
Post by: Datomas on October 06, 2020, 11:27:17 pm
I don't know if anyone still even uses the forums anymore, but...

 O:)
Title: Re: Sphere 5.5.1 (miniSphere)
Post by: Fat Cerberus on October 13, 2020, 08:35:21 am
Just thought I'd give a quick heads up, there's an effort underway to make Oozaru a first-class member of the Sphere toolkit. In a future version the plan is that you will continue to be able to test your game on miniSphere with full debugging support etc., but the final distribution target will be fully Oozaru-ready.
Title: Re: Sphere 5.5.2 (miniSphere)
Post by: Fat Cerberus on December 03, 2020, 10:46:11 am
I'm currently working on TypeScript integration for Sphere 5.6. This will make it so cell init creates a TS-enabled project. Initial experiments are going well, though there remain some kinks to work out. It doesn't seem like it will be too painful though, and most of the work was already done with my earlier ts-sphere (https://github.com/fatcerberus/ts-sphere) project. The goal is to get it to behave as much like native tsc as possible, without the need to install Node.js or npm first. Just a tsc() call in your Cellscript.
Title: Re: Sphere 5.5.2 (previously miniSphere)
Post by: Fat Cerberus on December 17, 2020, 12:59:46 am
The rebranding of miniSphere to Sphere is now fully complete! https://github.com/fatcerberus/sphere

The engine itself has also been renamed, to neoSphere.  But unlike with miniSphere, that name no longer takes center stage: It's simply the name of the engine implementation, to differentiate it from other implementations of the Sphere API such as Oozaru.
Title: Re: Sphere 5.6.2 (neoSphere, Cell, SSj)
Post by: Fat Cerberus on July 21, 2021, 03:30:58 pm
For anyone who still cares: Sphere 5.6.2 is available for both Windows and macOS.
Title: Re: Sphere 5.6.2 (neoSphere, Cell, SSj)
Post by: mezzoEmrys on July 22, 2021, 02:43:36 am
Stellar work once again!
Title: Re: neoSphere 5.6.4 (with Cell, SSj)
Post by: Ma3vis on December 02, 2021, 11:28:41 pm
So... I downloaded the latest SphereStudio and I can't find the map editor or sprite editor or anything really. I see there's a project tree and a task manager but I don't understand any of it or what's it for? Should I still use the older version of Sphere?
Title: Re: neoSphere 5.6.4 (with Cell, SSj)
Post by: Fat Cerberus on December 03, 2021, 01:19:43 am
Unless you somehow managed to disable all the plugins, the map editor, etc. should be there, you just have to click New -> Map, etc.  If you open an existing Sphere 1.x project with it you can also edit the maps, spritesets, etc. that are there (that's what the project tree is for).

I don't recommend using the old editor if you want to take advantage of the new API in neoSphere, but for what it's worth I am actively working on improving the user experience in Sphere Studio, there are admittedly some rough edges and I'd like to make things more seamless.
Title: Re: neoSphere 5.7.0
Post by: Fat Cerberus on December 14, 2021, 10:55:57 pm
neoSphere 5.7.0 has been released:
https://github.com/spheredev/neosphere/releases/tag/v5.7.0
Title: Re: neoSphere 5.8.0
Post by: Fat Cerberus on March 25, 2022, 10:38:28 am
5.8.0 is out, adding static functions for directly constructing basic transforms, e.g. Transform.scale(2.0, 2.0) returns a 2x scaling matrix, skipping the new Transform() step.
Title: Re: neoSphere 5.8.1
Post by: Datomas on December 07, 2022, 06:55:42 pm
Neato.
Title: Re: neoSphere 5.9.0
Post by: Fat Cerberus on November 16, 2023, 03:13:51 am
the next release will add a bunch of eaty pigs that eat all your code after you write it so maybe be prepared for that

also the pigs will increase the size of the download by approximately 812pB[1]because pigs are super morbidly obese and also fat
1 pigabyte = *MUNCH*
Title: Re: neoSphere 5.9.1
Post by: Fat Cerberus on December 14, 2023, 02:32:29 am
if anyone's wondering why the 5.9.1 release is missing the eaty pigs I promised it's because of an unexpected pig shortage... basically what happened is at the last minute all the pigs ate each other before I could get the release out.  on the plus side there were literally hundreds of pigs and someone apparently slipped them Red Bull so it was really, really funny to watch[1]
but not nearly as funny as the farmer who sucked up several pigs with a straw and then exploded (https://sdrodrian.com/poems/492.htm)
Title: Re: neoSphere 5.9.2
Post by: Radnen on January 02, 2024, 12:58:08 am
You've done fantastic work, Fat Cerberus! I do try to log in once every month or so. With the rise of so many indie games, I'm still surprised Sphere hasn't grown too much... I first found out about it on... sourceforge! Believe it or not, haha. I'm just glad to see miniSphere still gets some work on it every once and a while.
Title: Re: neoSphere 5.9.2
Post by: Fat Cerberus on January 02, 2024, 03:08:49 am
I'm just glad to see miniSphere still gets some work on it every once and a while.

Yeah, I don't have a lot of motivation to do feature work anymore, now that the engine is feature-complete[1] (I'm primarily motivated by necessity, which makes hobbies hard), but I do try to keep up with fixing bugs that people find.

That said, I'm quite proud of accomplishing the initial goal I set when I started working on neoSphere which was to make the "true" Sphere 2.0, backwards compatibility and all.  I probably went above and beyond on that, to a fault even - in that the v1 and v2 APIs aren't mutually exclusive and it's completely possible to mix them in the same codebase.
This even extends to Oozaru, as despite technically not being feature-complete yet, enough is implemented to run Specs so it might as well be...