Spherical forums

Sphere Development => Sphere General => Topic started by: Fat Cerberus on August 04, 2016, 03:04:42 am

Title: Sphere v2 API discussion
Post by: Fat Cerberus on August 04, 2016, 03:04:42 am
The release of minisphere 4.0 is imminent.  This version will introduce a brand-new API, dubbed "Sphere v2".  Sphere v2 is a complete reimagining of the Sphere API and I'd like to get some input on the different aspects of it before it's finalized in minisphere 5.0, locking us into backwards compatibility.

Note that Sphere v1 functions will still be available for the foreseeable future and can be freely used side-by-side with the new API, easing some of the pain of migration for existing large Sphere codebases.

I'll post more here in the coming days, but to kickstart the discussion, here is the API documentation for Sphere v2 as it currently exists in the minisphere Git repository:
https://github.com/fatcerberus/minisphere/blob/master/docs/sphere2-core-api.txt
Title: Re: Sphere v2 API discussion
Post by: Flying Jester on August 04, 2016, 03:23:38 pm
Just skimming, a few ideas:



Title: Re: Sphere v2 API discussion
Post by: Fat Cerberus on August 04, 2016, 04:18:56 pm
I agree re: resolution in JSON, I can change that before release.  I originally likes the elegance of representing resolution as a single string, but it's error prone this way, particularly if the json file is made by hand.

kb.getString() is there, except I named it kb.keyString() instead.  Not sure which name is better, probably getString.

I'm on the fence about engine.time.  On the one hand it saves some typing, but it is a little odd that if it occurs more than once on the same line of code it can end up representing different values (since it's usually sub-microsecond precision).  Ultimately you're probably right that it should be a function.

I'll look into vertex attributes, I didn't know that much about them so I left them out.

For Shape.draw(), it's worth noting that Node.js API is full of functions like this, where you can leave out one or more of the default arguments (even those in the middle) and it still works.  However in this case at least it is kind of awkward.  I'll see if I can come up with a better design.

What do you mean about a caveat for Socket.bytesPending?
Title: Re: Sphere v2 API discussion
Post by: Flying Jester on August 04, 2016, 05:59:19 pm

What do you mean about a caveat for Socket.bytesPending?


It isn't an API issue, but if you are writing documentation you should probably mention that this value can be stale by the time you actually perform a read. It's good to have people new to networking keep things like that in mind, and it's not bad to remind more experienced folks, too.


kb.getString() is there, except I named it kb.keyString() instead.  Not sure which name is better, probably getString.


Ah, I just didn't see it then.
Title: Re: Sphere v2 API discussion
Post by: Fat Cerberus on August 05, 2016, 01:22:31 am
Looking at Shape#draw() again, there is actually only one optional parameter--the transformation.  The surface is actually required; if you want to draw to the backbuffer, you just pass screen (which is a Surface).  So it should actually be okay the way it is.

One thing I want to highlight is the new RNG API.  I mentioned it in the minisphere thread, but didn't really go into detail on it.  Here's the API:
https://github.com/fatcerberus/minisphere/blob/master/docs/spherical-api.txt#L177-L221

It's based on the xoroshiro128+ generator and allows you to create multiple independently seeded random number generators.  Each RNG object exposes a state property which lets you save and restore the current position of that generator in its sequence.  This can be used to prevent save system abuse ("save scumming").  For example you could have an RNG which is used to determine the contents of random chests.  The state of this one would be stored in your save file so that you can't reload to get a different item.

Likewise, having independent generators is useful for preventing abuse.  A common exploit in older games is to generate a certain number of values before the action you wanted to manipulate in order to get a desired result.  By creating independent RNGs, this exploit can be mitigated, as the state of the RNG for item drops doesn't have to be affected by the one that determines, e.g. AI attacks.

For games which don't need this level of control over the RNG, the standard library provides the random module:
Code: (javascript) [Select]

const random = require('random');

// 25% chance this is true
console.log(random.chance(0.25));

// integer [1,10]
console.log(random.discrete(1, 10));

// expected value 1000, std. dev. 50
console.log(random.normal(1000, 50));

// 10 alphanumeric characters
console.log(random.string(10));

// one of: pig, cow, ape
var array = [ "pig", "cow", "ape" ];
console.log(random.sample(array));

Title: Re: Sphere v2 API discussion
Post by: Fat Cerberus on August 05, 2016, 10:08:56 am
A useful addition in Sphere v2 is the assert() function, which lets you add debug checks to your game that only have an effect under SSJ:

Code: (javascript) [Select]

maggie.eat(allOfEverything);
assert(maggie.weight >= 812, "maggie isn't fat enough");
maggie.sitOn(scott);


While minisphere is under SSJ control, a failing assert will trigger a warning message.  The user then has the option to either continue execution, or break into the debugger immediately.
Title: Re: Sphere v2 API discussion
Post by: Fat Cerberus on August 06, 2016, 01:03:00 pm
By the way, I should mention that Sphere v2 is kind of a two-pronged approach.  You have the low-level core API, and then the standard library ("miniRT"), which you pull in using require() includes versions of my Specs threader, Scenario, and even a prefab debug console:

https://github.com/fatcerberus/minisphere/blob/master/docs/miniRT-api.txt

The idea is that the standard library is guaranteed to exist on any Sphere v2 engine, so games are free to use its functionality the same as they would built-ins like Image, Color, etc.  Having Scenario as standard functionality will be very nice, I think. :)
Title: Re: Sphere v2 API discussion
Post by: Fat Cerberus on August 08, 2016, 02:42:47 am
This is kind of subjective thing, I'm mostly just curious which naming scheme you guys prefer for namespace-like "objects":

camelCase, initial lowercase
Code: (javascript) [Select]

ssj.trace("engine is " + system.name);
ssj.trace("mouse at x:" + mouse.x + ",y:" + mouse.y);
var start = system.now();
system.sleep(2.0);
var end = system.now();
ssj.trace("slept for " + (end - start) + " secs");


Pascal case, initial capital
Code: (javascript) [Select]

SSJ.trace("engine is " + System.name);
SSJ.trace("mouse at x:" + Mouse.x + ",y:" + Mouse.y);
var start = System.now();
System.sleep(2.0);
var end = System.now();
SSJ.trace("slept for " + (end - start) + " secs");


The latter seems less likely to cause naming conflicts, but the former feels more idiomatic for JS and looks cleaner.

For what it's worth, I prefer the first style.

Note: In both cases, singletons like screen would remain lowercase, since that's an actual built-in global variable (a Surface in the case of screen).
Title: Re: Sphere v2 API discussion
Post by: Radnen on August 08, 2016, 11:29:26 pm
I go with Crockford and Crockford says "Capitalize your classes". So constructors definitely Pascal case and everything else camelCase.
Title: Re: Sphere v2 API discussion
Post by: Fat Cerberus on August 11, 2016, 02:47:03 am
Yeah, I already capitalize classes such as Image, Shape, etc.  I was specifically referring to namespace-like objects, such as system.  It's not really clear-cut because it can either be seen as a pure namespace (like the JS Math object), in which case it should be capitalized for consistency, or an actual singleton object representing the system/engine, in which case camelCase makes more sense.

Moving on, I figure Sphere 1.x API support shouldn't be mandated in the v2 specification, but rather should be up to the engine implementor whether to support it or not.  As such, I added an entry to system.extensions - "sphere_api_v1_compatible" - to reflect that it's an extension to the standard v2 API.
Title: Re: Sphere v2 API discussion
Post by: Fat Cerberus on August 12, 2016, 02:58:10 pm
Should assert() and trace() ("print to debugger") be global functions?  In minisphere 4.0 they are referenced as e.g. ssj.assert() (i.e. a minisphere extension), but I can change them to global if anyone thinks this should be standard functionality.
Title: Re: Sphere v2 API discussion
Post by: Radnen on August 13, 2016, 03:25:43 am

Should assert() and trace() ("print to debugger") be global functions?  In minisphere 4.0 they are referenced as e.g. ssj.assert() (i.e. a minisphere extension), but I can change them to global if anyone thinks this should be standard functionality.


I would think they are more 'system calls' and so I think it's okay if they are global functions. Do you still have to import their library though?
Code: (javascript) [Select]

require('assert');

assert();


Or does it not work like that for global calls? Must they always be namespaced? Because while I like the idea of making assert global, there might be a time I don't use it.
Title: Re: Sphere v2 API discussion
Post by: Fat Cerberus on August 13, 2016, 03:33:22 am
Unlike Node, none of the core low-level API needs require().  What require() is for in Sphere is pulling in the standard library - Link, Scenario, Threads, etc.  The idea is that there's a core set of "system calls" which are provided by default and (eventually) standardized that all other modules and code are built on top of.  This is why a lot of the v2 API looks so barebones - games are meant to use the higher-level stuff like Scenario for the most part and only drop down to the low level when absolute full control is needed.
Title: Re: Sphere v2 API discussion
Post by: Radnen on August 13, 2016, 04:50:09 am
OK, that makes sense for a game engine then. Yeah, I'll accept having such system calls be global functions, for the sake of being "Sphere-y".

Edit:
That said, couldn't assert be a library? I think if system calls ought to be few, certainly assertions could be in a library with system logging/debugging calls being what the library would use anyways.
Title: Re: Sphere v2 API discussion
Post by: Fat Cerberus on August 13, 2016, 10:00:05 am

That said, couldn't assert be a library? I think if system calls ought to be few, certainly assertions could be in a library with system logging/debugging calls being what the library would use anyways.


The thing about assert() in minisphere is that it's already a pretty low-level call - if an assertion fails and SSJ is in use, then it displays an alert and triggers a breakpoint; without the debugger attached it does nothing at all.  This can only really be handled in native engine code, since the JS side has no way to tell if the debugger is in use (for good reason).  Likewise for trace - it doesn't have any effect if SSJ is not in use (as opposed to, say, console.log() which prints to the terminal).  There's not much a library can do to build on that, and would end up looking like this:

Code: (javascript) [Select]

exports.assert = function(test, msg) { system._assert(test, msg); }
exports.trace = function(msg) { system._trace(msg); }


Which of course doesn't gain us anything over just exposing the functions from the engine directly.
Title: Re: Sphere v2 API discussion
Post by: Fat Cerberus on August 13, 2016, 09:51:27 pm
I guess it would still be possible to build an "assert" module as part of the standard library though:
http://wiki.commonjs.org/wiki/Unit_Testing/1.0#Assert

One thing that could be done is to do the actual comparison in JS and just expand the SSJ API a bit to accommodate.  For example that "show error and break" could be its own function, `ssj.alert()` which would be a no-op if not debugging.

One thing I don't like about having an external library for assertions is that it increases the barrier to entry.  We want to encourage people to pepper their code with assertions, but if you have to import the assert module in every file that uses them, people might just get lazy and avoid doing it.  So it's not clear what the best setup would be.
Title: Re: Sphere v2 API discussion
Post by: Radnen on August 13, 2016, 10:27:00 pm

One thing I don't like about having an external library for assertions is that it increases the barrier to entry.  We want to encourage people to pepper their code with assertions, but if you have to import the assert module in every file that uses them, people might just get lazy and avoid doing it.  So it's not clear what the best setup would be.


I'm not sure. I for one never pepper my code with assertions. If there are to be assertions, in unit tests they go, not in actual game logic. But I guess it depends if you like a more contractual style or not. I'd rather add an assertion library to my unit test scripts instead.
Title: Re: Sphere v2 API discussion
Post by: Fat Cerberus on August 13, 2016, 10:47:52 pm
Unit tests test external behavior ("external" relative to the component under test, I mean).  It's my understanding that the other purpose of assertions is to verify assumptions made internally.  For example, maybe you wrote some code assuming a pointer (which is not visible to unit tests) won't be null because, if the program is well-behaved, it can't possibly be null.  If there's _any_ chance at all that the pointer could become null due to a regression elsewhere, you assert for it so that in a debug build, you crash as early as possible.  Duktape is actually loaded with asserts like that.

That's the reason why, in most programming languages, asserts don't actually do anything at all in a release build (they are usually compiled out completely) - because the scenarios they are designed for would cause massive slowdowns if they all had to be tested in the final build.  Unfortunately, JS doesn't have any kind of conditional compilation so we lose that benefit. :(

In any case, I'll implement that CommonJS Assert interface because it seems common enough in the wild (Firefox supports it, e.g.) that Sphere should have it too.
Title: Re: Sphere v2 API discussion
Post by: Radnen on August 14, 2016, 03:15:03 am
Yeah I was thinking it would be slower to pepper code with assertions, but then again in languages like C they can be compiled away.
Title: Re: Sphere v2 API discussion
Post by: Fat Cerberus on August 17, 2016, 12:10:26 pm
The one benefit to the Node.js-style assert library is that it actually throws an exception if the condition isn't met, making it useful for actual validation at runtime.  ssj.assert() never throws - it displays a warning and optionally triggers a breakpoint if the debugger is in use, but otherwise has no effect on the program flow.  Both have their uses.
Title: Re: Sphere v2 API discussion
Post by: Fat Cerberus on August 26, 2016, 11:18:39 am
I'm wondering what the best setup for font.getTextWidth/Height() is.  Right now I implemented it like Sphere 1.x, with the separate methods, but I wonder if it would be better to have a unified .getTextSize() function.  The function would look like

Code: [Select]
Font#getTextSize(text[, width]);


If you don't pass a width, it would just measures the width and height with no wrapping.  Otherwise it wraps the text to the specified with and gets the height of the block.

Do you think this would be more or less intuitive than the separate width/height functions we have now?
Title: Re: Sphere v2 API discussion
Post by: Fat Cerberus on September 05, 2016, 01:49:53 am
I'm still trying to hash out a good setup for the Sphere v2 joystick API.  Right now I'm thinking of a system where you access connected devices like this:

Code: (javascript) [Select]

var joy = Joystick.devices[0];
if (joy.isPressed(0)) { /* talk */ }
if (joy.getPosition(0) > 0.5) { /* move right */ }
// etc.


The important part is that:


Of course, this is just an idea and I'm free to change my mind.  Any opinions/ideas/suggestions?
Title: Re: Sphere v2 API discussion
Post by: Fat Cerberus on September 13, 2016, 12:00:04 pm
I've decided to drop the typed I/O (readInt, readString, writeFloat, etc.) from FileStream in favor of implementing the W3C Encoding API:
https://www.w3.org/TR/encoding/

This is more flexible, since it allows processing text in any ArrayBuffer or view, thus it'll be possible to encode and decode text from other sources such as sockets, which is currently not possible.  The API is a bit more convoluted than I'd prefer, but better to implement a standard rather than rolling my own API if there's no functionality to be gained by doing so.

Note: Reading ints and floats from an ArrayBuffer is already possible by using a DataView.
Title: Re: Sphere v2 API discussion
Post by: Fat Cerberus on October 31, 2016, 01:13:50 pm
Here's the new way to do update and render scripts coming in minisphere 4.3:
Code: (javascript) [Select]

var updateJob = system.recur(Dispatch.Update, myUpdateFunc);
var renderJob = system.recur(Dispatch.Render, myRenderFunc);


You can call recur() as many times as you want and it will keep adding jobs.  To remove them, just do:
Code: (javascript) [Select]

updateJob.cancel();
renderJob.cancel();


Best of all, this works engine-wide, not only in the map engine.
Title: Re: Sphere v2 API discussion
Post by: DaVince on October 31, 2016, 02:49:34 pm
That's kinda difficult to understand from a glance, but perhaps that's because I haven't given the new API stuff much of a look at all. (The original links 404 now too.) What is this about jobs again? How do you recur multiple things into a renderscript? Do you create separate jobs for that, like so?

Code: (javascript) [Select]

var updateJob = system.recur(Dispatch.Update, myUpdateFunc);
var anotherUpdateJob = system.recur(Dispatch.Update, someOtherFunc);
Title: Re: Sphere v2 API discussion
Post by: Fat Cerberus on October 31, 2016, 03:39:25 pm
Right, multiple calls to recur() create new hooks each time.  To cancel the job later, you just do updateJob.cancel().

It's like how setTimeout() works in web JS.  The first parameter to recur() specifies when to run it (update, render, on next tick) and the second is the function to call.
Title: Re: Sphere v2 API discussion
Post by: Fat Cerberus on October 31, 2016, 03:44:44 pm
By the way the reason I called them "jobs" is because that's how the ECMAScript specification refers to asynchronous calls performed from the event loop, as opposed to immediately.

Async is a bit hard to fully wrap your head around until you get used to it, so the confusion is understandable.
Title: Re: Sphere v2 API discussion
Post by: Fat Cerberus on November 01, 2016, 03:08:44 am
@DaVince: Would it make the API any easier to understand if I namespaced things a bit?  For example, something like this:

Code: (javascript) [Select]

// these are all recurring:
Dispatch.onUpdate(updateThis);
Dispatch.onUpdate(updateThat);
Dispatch.onUpdate(updateSomeOtherThing);
Dispatch.onFlip(renderItAll);

// these are one-shots:
Dispatch.now(resolvePromise);
Dispatch.later(2.0, getEatenBy.bind(you, maggie));


At first I thought "onUpdate" etc. might imply the job only runs once on the next update, but since Node.js uses a similar scheme (object.on('event', fn)), I don't think that should be a source of confusion in reality.

edit: Just saw you said the documentation links were 404s now.  I probably did some housekeeping and renamed them or something, here's the up-to-date links:
https://github.com/fatcerberus/minisphere/blob/master/docs/sphere2-api.txt
https://github.com/fatcerberus/minisphere/blob/master/docs/miniRT-api.txt

edit 2: Also added Dispatch.later() for making delayed calls.
Title: Re: Sphere v2 API discussion
Post by: DaVince on November 03, 2016, 02:47:04 pm
Hmm, that would certainly make them less obtuse to people reading the code. The code you posted before introduced a bunch of new things that would confuse people who are just getting into it. What you suggested does look a lot clearer!

I assume that even with the namespacing suggestion, you could still do job = Dispatch.onUpdate(updateThis); job.cancel()? :)
Title: Re: Sphere v2 API discussion
Post by: Fat Cerberus on November 03, 2016, 03:18:10 pm
Yep, that would still work. :D
Title: Re: Sphere v2 API discussion
Post by: Fat Cerberus on November 03, 2016, 03:45:06 pm
Also, and this is kind of a bikeshed issue, but I wonder if it would make more sense to have:

Code: (javascript) [Select]

job = Dispatch.onUpdate(...);
Dispatch.cancel(job);


The reason is that Job object is not really meant to be a real object but more of just an unforgeable "token" so that outside code can't accidentally cancel your job.  Using the token metaphor it's a bit strange to have the token do anything by itself - you need to "insert" it into the Dispatch machine for it to work.

setTimeout() also works like this, so there's that too. ;)
Title: Re: Sphere v2 API discussion
Post by: Fat Cerberus on November 04, 2016, 02:46:02 am
Final API design for the Dispatch API in minisphere 4.3 will probably look like this:

Code: (javascript) [Select]

// Call from event loop ASAP:
jobToken = Dispatch.now(func);

// Call from event loop in X seconds:
jobToken = Dispatch.later(2.0, function() { pig.eat(you); });

// Call at the start of every frame:
jobToken = Dispatch.onUpdate(function() { pig.hunger++; });

// Call at the end of every frame (just before flip).  These can have priority and
// higher priority jobs are rendered later in a frame according to the painter's
// algorithm:
jobToken = Dispatch.onRender(function() { pig.draw(); });
jobToken = Dispatch.onRender(function() { pig.draw(); }, 8);  // priority 8.0

// Cancel an active job:
Dispatch.cancel(jobToken);


Thoughts?
Title: Re: Sphere v2 API discussion
Post by: DaVince on November 04, 2016, 07:34:49 am
Looks good to me, though it seems sensible to me that Dispatch.later also has a version that works with frames rather than seconds. Also, does onUpdate also have priorities?
Title: Re: Sphere v2 API discussion
Post by: Fat Cerberus on November 04, 2016, 08:52:32 am
Good point, a lot of games need frame-perfect timing; since the engine manages the frame rate for you (tip: screen.frameRate) it makes sense to be able to do that.  I'll add it, thanks for the idea. :)

I could add priorities for onUpdate easily (the job queue is shared for all types), but I didn't originally expose it to the API because it seemed like a well-designed game wouldn't have any dependencies between different update scripts?  It seemed to matter more for rendering because render order affects the final scene.  Maybe there's a use case I'm not thinking of though.
Title: Re: Sphere v2 API discussion
Post by: DaVince on November 04, 2016, 08:55:27 am
Ohh, I see. It just seemed sensible to me that updates can also be executed in some specific order, just like renderscripts. Like updating the player position before or after checking enemy interactions, for example. But i see why you would give renderscripts specifically an order - it determines what overlaps what after all.
Title: Re: Sphere v2 API discussion
Post by: Fat Cerberus on November 04, 2016, 09:00:58 am

Ohh, I see. It just seemed sensible to me that updates can also be executed in some specific order, just like renderscripts. Like updating the player position before or after checking enemy interactions, for example. But i see why you would give renderscripts specifically an order - it determines what overlaps what after all.


Good point, there may be dependencies after all.  I'll add update priorities then, that'll be a simple fix.
Title: Re: Sphere v2 API discussion
Post by: Fat Cerberus on November 04, 2016, 10:34:01 am
Alright, I added priority support for onUpdate calls in the latest Git build.
Title: Re: Sphere v2 API discussion
Post by: Fat Cerberus on November 04, 2016, 04:19:31 pm
I decided to make unthrottled framerate be given as Infinity in Sphere v2 rather than 0:

Code: (javascript) [Select]

screen.frameRate = 60;  // 60 fps
screen.frameRate = Infinity;  // unthrottled
screen.frameRate = 0;  // RangeError: invalid frame rate


Having implemented this and then looked back at the Sphere v1 API setup, I can see now that it was designed without fully understanding all the intricacies of JavaScript.  The semantics are those of a typical C API, which also explains all the global functions and lack of any constructors.  Luckily that also makes it very easy to support the two APIs side-by-side. ;)

In any case it feels good to finally be making a clean break from the old designs.
Title: Re: Sphere v2 API discussion
Post by: Fat Cerberus on November 06, 2016, 01:04:31 am
system.now() in the latest development build now returns the number of frames processed since engine start, rather than wall-clock time.  If needed, wall-clock time can already be retrieved in JavaScript via Date.now(), so it seems much more useful for the Sphere API to work in frames, as that is usually the dominant timing method for realtime games.  That ensures timing remains precise even if you hit lag/slowdown.

Specifically, system.now() counts updates, not flips.  This way skipped frames don't throw off your timing.
Title: Re: Sphere v2 API discussion
Post by: Fat Cerberus on November 06, 2016, 09:54:49 am
Following up on that infinite framerate thing, I just realized that it's very common to do seconds * fps to figure out how many frames something will take.  This raises the issue of infinity being used to turn off the frame limiter: Because infinity * anything = infinity, such an operation would never complete.  Using 0 to disable the frame limiter like Sphere 1.x might actually be better, if somewhat less intuitive.
Title: Re: Sphere v2 API discussion
Post by: Fat Cerberus on January 23, 2017, 11:23:19 am
I decided to rename FS.mkdir() etc. to make them more discoverable.  I originally used the POSIX names to match Node, but one of Sphere's strengths is having very descriptive API names even at a glance, so I made the following renames:

Title: Re: Sphere v2 API discussion
Post by: DaVince on February 08, 2017, 04:57:39 am
I'm looking at the sphere2-api.txt doc right now, and was thinking: how about a screen.toggleFullScreen() method to, well, toggle the fullscreen mode? Perhaps it could even take a boolean argument to force full screen on or off.

Also, did the Sphere 2 project files have any setting to define a game as being full screen or windowed by default? I kinda forgot. Also, are these settings writable by the game itself in an intuitive manner?
Title: Re: Sphere v2 API discussion
Post by: Fat Cerberus on February 08, 2017, 10:58:03 am
None of that is supported yet, but all good ideas :).  This is why system.apiLevel is still 0, I'm still open to making modifications!

Full screen toggle could just be a property, no function needed:
Code: (javascript) [Select]

screen.fullScreen = !screen.fullScreen;


That'd be easy to implement too.
Title: Re: Sphere v2 API discussion
Post by: Fat Cerberus on February 09, 2017, 02:32:51 am
Alright, I added a screen.fullScreen property.  You'll be able to set it to true or false to control the engine's fullscreen mode at runtime.

I reorganized the file API a bit: Instead of FS.openFile() (which is awkward to explain to newbies: what does it mean to "open" a file?  It's not a folder.), you now do instead:

Code: (javascript) [Select]

import { FileReader, FileWriter } from 'struct';

function saveGame(world)
{
    var stream = new FileStream("~/saveFile.dat", FileMode.Write);
    var fw = new FileWriter(stream);
    fw.writeString8(world.playerName);
    // etc. etc.
}


There are no more obscure C fopen() mode strings, it uses easy-to-understand FileMode constants now.  And while it seems superficial, the rename from FS.openFile() to new FileStream() is actually pretty significant: It takes the concept of "opening a file" completely out of the equation.  That makes it much simpler to explain to new users: you create a new FileStream which provides streaming access to the data stored in the file.

This kind of thing is a delicate balance to strike: You want the code calling these API functions to be self-documenting (so that less experienced users can read it and see what's going on, and thus learn from it), but at the same time you don't want to dumb it down and take power away from more advanced users.  That sometimes requires a lot of thought, but in the end I think the work has paid off.  For many, Sphere 1.x served as a "gateway drug" into programming.  I think I've been able to preserve that gateway potential in Sphere v2, but only time will tell.
Title: Re: Sphere v2 API discussion
Post by: Fat Cerberus on February 10, 2017, 02:50:58 am
5.0 will change the save data handling so that games no longer have to share the same save file folder.  How that works is you set a "save ID" in the JSON manifest and miniSphere uses that as the name of the folder.  If you have want to have sequels be able to share save data, you can just give them the same save ID.

I'm thinking there should be a standardized format for save IDs to avoid clashes, something like this maybe (inspired by Android app IDs):
Code: [Select]
com.authorname.gamename
Title: Re: Sphere v2 API discussion
Post by: DaVince on February 10, 2017, 03:38:13 am
I really like that idea. It opens up interaction with other games, sequels, or even stuff like save file managers.
Title: Re: Sphere v2 API discussion
Post by: Fat Cerberus on February 13, 2017, 01:34:50 pm

Also, are these settings writable by the game itself in an intuitive manner?


That's not currently possible.  It wouldn't be too hard to have the engine detect changes to Sphere.Game and write them back to the manifest, but I'm not sure what to do about SPK-packed games in that case.
Title: Re: Sphere v2 API discussion
Post by: mezzoEmrys on February 13, 2017, 06:07:29 pm
It does seem like it would be a desirable thing to have it so that if a user enables fullscreen mode, the game would be able to load up in fullscreen mode again the next time the game starts up, without the user having to turn it on again. Maybe there should be some kind of save file that contains "runtime selected" settings which can overwrite the default on load?
Title: Re: Sphere v2 API discussion
Post by: DaVince on February 14, 2017, 07:19:29 am
Good point. Perhaps it could save a copy of the project file in a special folder, with a system in place to ensure same-named projects don't conflict of course.
Title: Re: Sphere v2 API discussion
Post by: Fat Cerberus on February 15, 2017, 11:59:46 am
If anyone's curious what proper Sphere v2 code will look like compared to legacy, this should give a good idea:
https://github.com/fatcerberus/kh2bar/blob/master/src/main.js
Title: Re: Sphere v2 API discussion
Post by: DaVince on February 16, 2017, 11:02:39 am
Whoa. It's really interesting to see no initializing function, and to see every script basically have its own "global"-like scope for itself. That actually seems really good to avoid conflicts because importedfunctiona.property will never clash with importedfunctionb.property.

Thank you for this example, really. It certainly answers some questions for me, just having some sample main script like this. :)
Title: Re: Sphere v2 API discussion
Post by: Fat Cerberus on February 18, 2017, 11:41:42 am
I'll probably need to rename Image to Texture to maintain cross-compatibility with Oozaru, because the browser platform already includes an Image class which is incompatible with Sphere.  The rename probably makes sense anyway, since it aligns with how images are used in Galileo (i.e. you don't draw them directly, they are used to texture shapes).
Title: Re: Sphere v2 API discussion
Post by: casiotone on March 22, 2017, 09:46:38 am

5.0 will change the save data handling so that games no longer have to share the same save file folder.  How that works is you set a "save ID" in the JSON manifest and miniSphere uses that as the name of the folder.  If you have want to have sequels be able to share save data, you can just give them the same save ID.

I'm thinking there should be a standardized format for save IDs to avoid clashes, something like this maybe (inspired by Android app IDs):
Code: [Select]
com.authorname.gamename


Could this not be set at runtime? A game may want to have its own save directory but still be able to access previous game data.

Perhaps the game ID could be set in the manifest, but an optional one also at runtime - and enforcing read-only access if it isn't the ID in the manifest.
Title: Re: Sphere v2 API discussion
Post by: Fat Cerberus on March 22, 2017, 09:53:29 am
That's a good point; games may also want to access another game's data that it wouldn't necessarily need to share a save directory with.  I like the idea of enforcing read-only access too.  I'll add an issue on GitHub about it, thanks for the idea. :)
Title: Re: Sphere v2 API discussion
Post by: Fat Cerberus on June 03, 2017, 10:12:53 am
New Sphere v2 Sample object (analogue to the v1 SoundEffect API):
https://github.com/fatcerberus/minisphere/blob/master/docs/sphere2-api.txt#L1001-L1047

In the spirit of reusing a sample for multiple playbacks of a sound effect, the playback parameters (pitch, speed, volume) get passed as options to play() instead of being associated with the Sample object.  I always found the way Sphere v1 handled that to be weird and unintuitive, never sure whether setting a value would apply to currently playing instances of a sound or only future playbacks (spoiler: it's the latter).  At least this way the API is very clear.
Title: Re: Sphere v2 API discussion
Post by: Rhuan on June 03, 2017, 01:02:53 pm

New Sphere v2 Sample object (analogue to the v1 SoundEffect API):
https://github.com/fatcerberus/minisphere/blob/master/docs/sphere2-api.txt#L1001-L1047

In the spirit of reusing a sample for multiple playbacks of a sound effect, the playback parameters (pitch, speed, volume) get passed as options to play() instead of being associated with the Sample object.  I always found the way Sphere v1 handled that to be weird and unintuitive, never sure whether setting a value would apply to currently playing instances of a sound or only future playbacks (spoiler: it's the latter).  At least this way the API is very clear.
Now that is an awesome feature I like the sound of - when are you planning the next miniSphere release to include it?
Title: Re: Sphere v2 API discussion
Post by: Fat Cerberus on June 03, 2017, 01:57:58 pm
I'm planning a 4.6 release soon, but there are a few more changes I want to work out first (such as the engine not installing on Ubuntu 16.10 and later).  Should be within the next couple of weeks if all goes well.  I was originally going to hold off on doing any more releases until 5.0, but there are a few annoying bugs that need to be fixed--Dispatch jobs rendering in the wrong order, for example--so an interim release is called for.
Title: Re: Sphere v2 API discussion
Post by: Rhuan on June 03, 2017, 04:24:11 pm

I'm planning a 4.6 release soon, but there are a few more changes I want to work out first (such as the engine not installing on Ubuntu 16.10 and later).  Should be within the next couple of weeks if all goes well.  I was originally going to hold off on doing any more releases until 5.0, but there are a few annoying bugs that need to be fixed--Dispatch jobs rendering in the wrong order, for example--so an interim release is called for.

Ok I've got a few other things on as well at the moment as well, I'll plan on building a mac version of 4.6 and uploading it.
Title: Re: Sphere v2 API discussion
Post by: Fat Cerberus on June 24, 2017, 10:39:54 am
If anyone wants to see what a real Sphere v2 game will look like, see the code for my kh2Bar showcase.
https://github.com/fatcerberus/kh2bar

The demo uses the engine's event loop rather than running its own and thus should be compatible with Oozaru in the future too.

Note that the code will only run on miniSphere 4.6, which is not released yet and requires building from source.  The release is almost done, but there are a couple loose ends I have to tie up (like Ubuntu 16.10+ compatibility).
Title: Re: Sphere v2 API discussion
Post by: Fat Cerberus on July 04, 2017, 12:55:09 am
miniRT is now called Sphere Runtime and is an official part of Sphere v2.  Here's what the API for it will look like in miniSphere 4.6:
https://github.com/fatcerberus/minisphere/blob/master/docs/sphere2-hl-api.txt
Title: Re: Sphere v2 API discussion
Post by: Fat Cerberus on July 06, 2017, 02:58:51 am
For transform.project3D() (to create a perspective projection), should the function work like glFrustum() or gluPerspective()?  The former lets you put the viewport in any orientation you want but is less intuitive to work with; while the latter accepts more human-readable parameters: field of view angle and aspect ratio, but forces you to use an on-axis projection (i.e. the camera points at 0,0 in clipspace always)
Title: Re: Sphere v2 API discussion
Post by: Rhuan on July 07, 2017, 07:35:51 pm
1. In answer to your question: I'd go with glFrustrum, can always add a helper function to simplify if needed.

2. I was trying to use logger.js earlier but couldn't get it to work:
const logger = require("logger");

var log = new logger.Logger("some file");
-> cannot construct undefined

3. I tried copy and pasting the code into my script then ran into problems with read/write access - the "user safe file directory" was not defined so I couldn't use ~/ and the game's folder isn't writeable so I couldn't use @/ or just a path without a prefix.

4. I've being doing some tests with galileo, now some of what it can do is great BUT, I tried having it run brownian motion and it seems slower than using transformBlit. I'm talking about drawing 400-1000 rings on screen, each one spinning (deformation of shape), moving in x-y, (translation) and moving in z (scaling).

With Galileo I tried doing these as:
(results based on modelling brownian motion with 800 rings on screen)
a) an array of models each with 4 vertices, textured with an image of a ring, one Model for each deformation, then drawing by picking the right model to represent each ring and setting it's transformation to the correct scale and translation. Result: 35-40 FPS
b) one model with 4 vertices, textured with an image of a ring, then for each ring setting it's matrix to represent the deformation and the scale and translation. Result: 35-40 FPS
c) one model with ~100 vertices, no texture, then for each ring setting it's matrix to represent the deformation and the scale and translation.Result: 40-44 FPS

With Sphere v1's api:
I used an image of a ring and used transformBlit to draw it with the necessary deformation, scale and translation, Result: 60-61 FPS

I note that all of the above tests were with the framerate set to 60. If I use an unlimited framerate it would go up to ~100 even with the slower Galileo methods, transform blit still would score higher but it's not so relevant at that point, this implies there is something wrong with the way minisphere limits a framerate in that it results in a significant extra slow down
Title: Re: Sphere v2 API discussion
Post by: Fat Cerberus on July 07, 2017, 07:57:35 pm

2. I was trying to use logger.js earlier but couldn't get it to work:
const logger = require("logger");

var log = new logger.Logger("some file");
-> cannot construct undefined


The Sphere Runtime API got overhauled and simplified in miniSphere 4.6.  See documentation:
https://github.com/fatcerberus/minisphere/blob/v4.6.0/docs/sphere2-hl-api.txt#L561-L591

For the lazy ;)
Code: (javascript) [Select]

const Logger = require('logger');
var log = new Logger('~/logFile.txt');


Quote

3. I tried copy and pasting the code into my script then ran into problems with read/write access - the "user safe file directory" was not defined so I couldn't use ~/ and the game's folder isn't writeable so I couldn't use @/ or just a path without a prefix.


Using ~ requires you to set a saveID in the game's JSON file.  If you still need to use the Sphere 1.x editor you can have an SGM and JSON file at the same time and miniSphere will prefer the latter.

Quote

4. I've being doing some tests with galileo, now some of what it can do is great BUT, I tried having it run brownian motion and it seems slower than using transformBlit. I'm talking about drawing 400-1000 rings on screen, each one spinning (deformation of shape), moving in x-y, (translation) and moving in z (scaling).

With Galileo I tried doing these as:
(results based on modelling brownian motion with 800 rings on screen)
a) an array of models each with 4 vertices, textured with an image of a ring, one Model for each deformation, then drawing by picking the right model to represent each ring and setting it's transformation to the correct scale and translation. Result: 35-40 FPS
b) one model with 4 vertices, textured with an image of a ring, then for each ring setting it's matrix to represent the deformation and the scale and translation. Result: 35-40 FPS
c) one model with ~100 vertices, no texture, then for each ring setting it's matrix to represent the deformation and the scale and translation.Result: 40-44 FPS

With Sphere v1's api:
I used an image of a ring and used transformBlit to draw it with the necessary deformation, scale and translation, Result: 60-61 FPS

I note that all of the above tests were with the framerate set to 60. If I use an unlimited framerate it would go up to ~100 even with the slower Galileo methods, transform blit still would score higher but it's not so relevant at that point, this implies there is something wrong with the way minisphere limits a framerate in that it results in a significant extra slow down


Hmm... this one is difficult to diagnose without actually seeing your code.  I have a sneaking suspicion it might be due to all the shader and matrix changes I mentioned in the miniSphere thread, though.  It definitely shouldn't be slower, that needs to get fixed ASAP ;)
Title: Re: Sphere v2 API discussion
Post by: Rhuan on July 07, 2017, 08:10:29 pm
Thanks for the replies.

File I/O
I can't see anything in the documentation about needing a .json manifest in order to have a save file directory available, that said I am looking at migrating everything to v2 (when I'm happy with each part of v2 working properly) but I definitely want to be able to write to my game's directory - I want to be able to do weird setup scripts for one :p and I also want to be able to keep save data with the game rather than somewhere else, so I think I'll have to stick to the v1 file I/O stuff unless you can add an option for that in future.

Galileo
I note I tried the Galileo tests with my custom shader applying colour masks and also with no shader specified and there was no significant change in speed.

The impact of using FlipScreen and no SetFrameRate call vs either calling SetFrameRate or using screen.flip was pretty significant though. Target frame rate of 60 and it would only mange to do 35-45, remove the target and suddenly it could do 70+ even 100 in some of the tests.
Title: Re: Sphere v2 API discussion
Post by: Fat Cerberus on July 07, 2017, 08:26:18 pm

File I/O
I can't see anything in the documentation about need a .json manifest in order to have a save file directory available, that said I am looking at migrating everything to v2 (when I'm happy with each part of v2 working properly) but I definitely want to be able to write to my game's directory - I want to be able to do weird setup scripts for one :p and I also want to be able to keep save data with the game rather than somewhere else, so I think I'll have to stick to the v1 file I/O stuff unless you can add an option for that in future.


The game directory is read-only for consistency more than anything.  If you make an SPK package, the engine is unable to write to the SPK and needs to redirect the write elsewhere (i.e. into the user's home directory, like where ~/ points).  Sphere v1 does exactly that; however I suspected it'd be confusing for less experienced users that, if they make a loose directory, the files get saved with the game, but if it's an SPK, the changes are virtualized and are lost when the SPK is copied to a different machine.  And there's also the issue of Cell being the preferred workflow for Sphere v2: It's natural when using a build tool to nuke the build directory every once in a while, I do this quite often myself with miniSphere, especially to get rid of any detritus before a release.  If your game is writing data to the game directory and you do that, all your test saves and other such data get wiped too.  Enforcing save data in ~/ is inconvenient, but avoids mishaps.

Overall, I decided when I was designing SphereFS that the way it's implemented now is for the best.  The sandboxing is annoying for more advanced users, but it prevents novice and intermediate users from shooting themselves in the foot.

Quote

Galileo
I note I tried the Galileo tests with my custom shader applying colour masks and also with no shader specified and there was no significant change in speed.

The impact of using FlipScreen and no SetFrameRate call vs either calling SetFrameRate or using screen.flip was pretty significant though. Target frame rate of 60 and it would only mange to do 35-45, remove the target and suddenly it could do 70+ even 100 in some of the tests.


Your graphics driver might be enforcing vsync, which if the game is already chugging, will just make the situation worse.  In any case I'll look into it when I get a chance.
Title: Re: Sphere v2 API discussion
Post by: Fat Cerberus on July 07, 2017, 08:30:31 pm
You're right, by the way, that saveID isn't mentioned in the documentation.  I added it to the release notes for miniSphere 4.5, but it seems I never documented it.  Sorry for the inconvenience! :-[
Title: Re: Sphere v2 API discussion
Post by: Fat Cerberus on July 08, 2017, 10:21:58 am

I can't see anything in the documentation about needing a .json manifest in order to have a save file directory available


I fixed that, see:
https://github.com/fatcerberus/minisphere/blob/master/docs/sphere2-core-api.txt#L80-L89

:)

saveID wasn't required before miniSphere 4.5--all games shared the same save directory.  But that made it difficult to write library code that used the save store.
Title: Re: Sphere v2 API discussion
Post by: Rhuan on July 08, 2017, 11:16:10 am
I still want the ability to edit documents in the game's folder from within the api - maybe after setting a special mode or something to avoid mistakes? (One practical use is the ability to change the game's default screen res you ought to be able to edit the game's manifest to do that, but there are a stack of other uses too)


Anyhow, for now the graphics speed issue, I've cleaned up my script so you can run and it see what you think. couple of notes: I switched the v1 function from transformBlit to transformBlitMask and now it's speed is closer to the v2 implementations - though it's still slightly faster conversely the v2 versions don't seem to have any speed difference for having a custom shader that does masking vs not having one.

Sorry about some of the variable names, was in an odd mood when writing this, also I used tab and space interchangeably so some of the indenting is a bit weird, should look normal if pasted into the sphere editor or any text editor where tab = 2 spaces.

Change the values set to the variables at the top to try out the different options, note I think modes 1 and 2 are just bad ideas and that the comparison should be between modes 0 and 3.

Code: [Select]
//what mode
//0 = shape + model per deformation, with 4 vertices per shape and one texture shared by all
//1 = one shape + model only but with 4 vertices and a texture
//2 = one shape + model only but with points (variable  below) number of vertices and no texture
//3 = v1 .transformBlitMask
var mode =0;
var points = 1000; // how many points on a ring
var num_rings = 1000; //how many rings to draw
var speed_limit = false; //true to use screen.flip (60 FPS limt) false to use FlipScreen - no limit

//load shader - set to Shader.Default if you don't have shaders to use
//note script below will pass shader 4 floats m_r, m_g, m_b and m_a for color masking
//tests show that using the Default shader (and hence no colours in modes 0,1 and 2 has no noticeable speed impact vs using custom vertex shader that masks
var shades = new Shader({vertex: "shaders/customised.vert.glsl", fragment: "shaders/customised.frag.glsl"});


//function to set up rings
function generate_rings(how_many,min_x,max_x,min_y,max_y)
{
  var rings = new Array();
  for(var i = 0; i < how_many; ++i)
  {
    rings[i]    = new Object();
    rings[i].x  = Math.floor(Math.random()*(max_x-min_x))+ min_x;
    rings[i].y  = Math.floor(Math.random()*(max_y-min_y))+ min_y;
    rings[i].z  = 0.6 - (2 * Math.random() / 5);
    rings[i].t  = Math.floor(Math.random()*100);
    rings[i].dx = Math.round(10 * (Math.random() - 0.5));
    rings[i].dy = Math.round(7  * (Math.random() - 0.5));
    rings[i].dz = 0.1  * (Math.random() - 0.5);
    rings[i].dt = 5 - Math.ceil(10 * Math.random());
    rings[i].col = [1,1,1,0.9];
    rings[i].dc  = [(1-Math.round(Math.random()))*Math.random()*0.01,(1-Math.round(Math.random()))*Math.random()*0.01,(1-Math.round(Math.random()))*Math.random()*0.01,(1-Math.round(Math.random()))*Math.random()*0.01];
  }
  return rings;
}

//function to update and draw the rings
function update_random_rings(rings,update,min_x,max_x,min_y,max_y)
{
  for(var i = 0; i < rings.length; ++i)
  {
    if(update)
    {
      if(rings[i].t + rings[i].dt > 99)
      {
        rings[i].t = 0;
      }
      else if(rings[i].t + rings[i].dt <0)
      {
        rings[i].t = 99;
      }
      else
      {
        rings[i].t += rings[i].dt;
      }
      rings[i].x += rings[i].dx;
      rings[i].y += rings[i].dy;
      rings[i].z += rings[i].dz;
      if (rings[i].x > max_x)
      {
        rings[i].dx = - Math.round(4 * Math.random());
      }
      else if (rings[i].x < min_x)
      {
        rings[i].dx =   Math.round(4 * Math.random());
      }
      else if(Math.random() > 0.99)
      {
        rings[i].dx -= Math.round(3 * (Math.random() - 0.5));
      }
      if (rings[i].y > max_y)
      {
        rings[i].dy = - Math.round(3 * Math.random());
      }
      else if (rings[i].y < min_y)
      {
        rings[i].dy =   Math.round(1.5 * Math.random());
      }
      else if (Math.random() > 0.99)
      {
        rings[i].dy -= Math.round(1.5 * (Math.random() - 0.5));
      }
      if (rings[i].z <0.2)
      {
        rings[i].dz = 0.02  * (Math.random());
      }
      else if (rings[i].z > 1.7)
      {
        rings[i].dz = - 0.02  * (Math.random());
      }
      else if (Math.random() > 0.99)
      {
        rings[i].dz -= 0.005  * (Math.random() - 0.5);
      }
      if(rings[i].dt > 4)
      {
        rings [i].dt = 2;
      }
      else if(rings[i].dt <-4)
      {
        rings [i].dt = -2;
      }
      else if(Math.random() >0.95)
      {
        rings[i].dt += 1 - Math.ceil(Math.random()*2)
      }
      rings[i].col[0] = (rings[i].col[0] + rings[i].dc[0]) % 1;
rings[i].col[1] = (rings[i].col[1] + rings[i].dc[1]) % 1;
rings[i].col[2] = (rings[i].col[2] + rings[i].dc[2]) % 1;
rings[i].col[3] = (rings[i].col[3] + rings[i].dc[3]) % 1;


    }
    if(mode < 3)
    {
      foo.identity();
   
      if(mode > 0)
      {
        foo.matrix[1][0] = 3 * (Math.sin(rings[i].t * Math.PI/45) - Math.cos(rings[i].t * Math.PI/45)) / 8;
        foo.matrix[1][1] = 0.5 - Math.sin(rings[i].t * Math.PI/45) - Math.cos(rings[i].t * Math.PI/45);
        foo.matrix[1][2] = 3 * Math.cos(rings[i].t * Math.PI/45);
        model_.setFloat("m_r", rings[i].col[0]);
        model_.setFloat("m_g", rings[i].col[1]);
        model_.setFloat("m_b", rings[i].col[2]);
        model_.setFloat("m_a", rings[i].col[3]);
      }
      else
      {
    models[rings[i].t].setFloat("m_r", rings[i].col[0]);
    models[rings[i].t].setFloat("m_g", rings[i].col[1]);
    models[rings[i].t].setFloat("m_b", rings[i].col[2]);
    models[rings[i].t].setFloat("m_a", rings[i].col[3]); 
      }
   
      foo.scale(rings[i].z / 2,rings[i].z /1.5);   
      foo.translate(rings[i].x, rings[i].y);  

   
  if(mode > 0)
  {
    foo.translate(0, 50 * Math.cos(rings[i].t * Math.PI/100)); 
    model_.draw(screen);
  }
  else
  {
        models[rings[i].t].draw(screen);
      }
    }
    else
    {
      jeff.transformBlitMask(rings[i].x ,
                             rings[i].y + (rings[i].z * 25)                    * Math.cos(rings[i].t * Math.PI/50),
                             rings[i].x +  rings[i].z * 80,
                             rings[i].y + (rings[i].z * 25)                    * Math.sin(rings[i].t * Math.PI/50),
                             rings[i].x +  rings[i].z * 80,
                             rings[i].y +  rings[i].z * 12.5 - rings[i].z * 25 * Math.cos(rings[i].t * Math.PI/50),
                             rings[i].x,
                             rings[i].y +  rings[i].z * 12.5 - rings[i].z * 25 * Math.sin(rings[i].t * Math.PI/50),CreateColor(rings[i].col[0]*255,rings[i].col[1]*255,rings[i].col[2]*255,rings[i].col[3]*255));

    }
  }
}

//create the ring image
function ring(colour_a, colour_b, a, b)
{
  if(mode < 3)
  {
    var output = new Surface(2*a + 2, 2*b + 2, new Color(0,0,0,0)); 
  }
  else
  {
    var output = CreateSurface(2*a+2,2*b+2,CreateColor(0,0,0,0));
  }
 
  var _vertices = [];
  for(var i = 0, t_x = 0, t_y = 0; i < (points/5); ++i)
  {
    t_x = a * Math.cos(i*Math.PI/(points/10)) + 1;
    t_y = b * Math.sin(i*Math.PI/(points/10)) + 1;
   
    _vertices.push({x:t_x,y:t_y,color:colour_a});
    _vertices.push({x:t_x + 1,y:t_y,color:colour_b});
    _vertices.push({x:t_x,y:t_y + 1,color:colour_b});
    _vertices.push({x:t_x -1,y:t_y,color:colour_b});
    _vertices.push({x:t_x,y:t_y-1,color:colour_b});
  }
 
  if(mode <3)
  {
    foo.translate(a,b);
    var temp = new Shape(_vertices,null,ShapeType.LineLoop);

    temp.draw(output,foo);
    if(mode < 2)
    {
      return output.toTexture();
    }
    else
    {
      return new Model([temp],shades);
    }
  }
  else
  {
    for(i = 0; i < _vertices.length; ++i)
    {
      output.setPixel(_vertices[i].x + a, _vertices[i].y + b, CreateColor(_vertices[i].color.r*255,_vertices[i].color.g*255,_vertices[i].color.b*255));
    }
    return output.createImage();
  }
}




var fun_rings =[];

fun_rings[0] = generate_rings(Math.floor(num_rings/4),0,400,0,357);
fun_rings[1] = generate_rings(Math.floor(num_rings/4),0,400,377,748);
fun_rings[2] = generate_rings(Math.floor(num_rings/4),600,1024,0,357);
fun_rings[3] = generate_rings(Math.floor(num_rings/4),600,1024,377,748);


var coords = [[50,400,20,357],[50,400,377,730],[500,1000,377,730],[500,1000,0,357]];



var foo = new Transform();

var jeff = ring(new Color(1,1,1,1),new Color(0.9,0.9,0.9,1),65,30);


//create the models for the rings type of model determined by mode
if (mode == 0)
{
  var shapes = [];
  var models = [];

  var y_s =[];
  var v_s = [];
  for (var inc = 0; inc < 100; ++inc)
  {
    y_s[0] = Math.round(Math.cos(inc * Math.PI/50) * 30);
    y_s[1] = Math.round(Math.sin(inc * Math.PI/50) * 30);
    y_s[2] = Math.round(15 - Math.sin(inc * Math.PI/50) * 30);
    y_s[3] = Math.round(15 - Math.cos(inc * Math.PI/50) * 30);

    shapes[inc] = new Shape([{x:0, y:y_s[0],u:0,v:0,color:Color.White},
                             {x:80,y:y_s[1],u:1,v:0,color:Color.White},
                             {x:0, y:y_s[2],u:0,v:1,color:Color.White},
                             {x:80,y:y_s[3],u:1,v:1,color:Color.White}],jeff,ShapeType.TriStrip);
                            


    models[inc] = new Model([shapes[inc]],shades);
    models[inc].setInt("mask_mode", 1);
    models[inc].transform = foo;
         
  }
}
else if (mode == 1)
{
  var shape_ = new Shape([{x:0,y:0,u:0,v:0,color:Color.White},
                             {x:80,y:0,u:1,v:0,color:Color.White},
                             {x:0,y:30,u:0,v:1,color:Color.White},
                             {x:80,y:30,u:1,v:1,color:Color.White}],jeff,ShapeType.TriStrip);
  var model_ = new Model([shape_],shades);
  model_.setInt("mask_mode", 1);
  model_.transform = foo;
}
else if(mode == 2)
{
  model_ = jeff;
  model_.setInt("mask_mode", 1);
  model_.transform = foo;
}


//loop to draw rings etc
var time = GetTime();
var time1 = GetTime();
var flag = false;
var ticker = 0;
while(!IsKeyPressed(KEY_ESCAPE))
{
  if(time + 25 < GetTime())
  {
    time = GetTime();
    flag = true;
  }
  if(time1 + 2000 < GetTime())
  {
    time1 = GetTime();
    if(Math.random() > 0.7)
    {
      ++ticker;
    }
  }
  for(i=0;i<4;++i)
  {
    update_random_rings(fun_rings[i],flag,coords[(ticker + i) % 4][0],coords[(ticker + i) % 4][1],coords[(ticker + i) % 4][2],coords[(ticker + i) % 4][3])
  }
flag = false;
while(IsAnyKeyPressed()&&(!IsKeyPressed(KEY_ESCAPE)));
  if(speed_limit)
  {
  screen.flip(); 
  }
  else
  {
    FlipScreen();
  }
}
Title: Re: Sphere v2 API discussion
Post by: Fat Cerberus on July 08, 2017, 11:36:57 am

One practical use is the ability to change the game's default screen res you ought to be able to edit the game's manifest to do that


This is a popular Sphere v1 hack, however Sphere v2 lets you directly change screen.width and screen.height at runtime to set the resolution - and you don't need to restart the game!  I'd be iffy of self-modifying games, as that's an easy way to destroy data just in the course of normal testing.  And if you're using Cell or some other build process, you'd still need to write your code with the assumption that the modifications aren't persistent - because you might at some point decide to blow away the build directory (and if you're using git, the build directory wouldn't be committed at all).  It's just a really easy way to introduce bugs all around and I'd prefer to avoid leaving such a dangerous footgun lying around.

That said, I'd probably be willing to implement some sort of "dev mode" that could be enabled in the manifest which would allow things like absolute paths, write access to the game directory, etc. that are normally denied.  However the default would remain the strict sandboxed mode, and I would heavily recommend to not enable devmode for redistributed games.

Anyway I'll look at the graphics code to see if I can find what's wrong.

By the way: screen.flip() can be unthrottled by setting
Code: (javascript) [Select]

screen.frameRate = Infinity;
Title: Re: Sphere v2 API discussion
Post by: Fat Cerberus on July 08, 2017, 09:18:14 pm
@Rhuan: I posted a GitHub issue about allowing a writable game directory so I don't forget all about it in a week ;)
https://github.com/fatcerberus/minisphere/issues/183
Title: Re: Sphere v2 API discussion
Post by: Fat Cerberus on July 08, 2017, 11:12:09 pm
I tested your code; Galileo is indeed significantly slower.  Results when using Shader.Default with speed_limit = false and vsync disabled, by mode:


Considering that transformBlitMask() does this under the hood:
https://github.com/fatcerberus/minisphere/blob/v4.6.0/src/minisphere/vanilla.c#L3624-L3633

And also considering the fact that I have an overclocked GTX 1070 in this machine, 90 FPS for something this simple is abysmal.  Something is clearly wrong with my Galileo implementation.  I'll need to do some profiling to figure out what's causing the performance issues.

edit:
Galileo Model#draw() under the hood:
https://github.com/fatcerberus/minisphere/blob/v4.6.0/src/minisphere/galileo.c#L200-L247

One thing that jumps out at me immediately is that 1) Each model drawn does two shader changes, once to switch to the model's shader and again to restore the v1 shader; 2) Each model drawn does four matrix uploads: twice each (i.e. set, draw, restore) for the modelview and projection matrices; and 3) Uniforms are sent each time the model is drawn.  The Sphere v1 call does none of this because it can safely assume the correct settings are already in place, largely thanks to the above shenanigans. :P

Clearly this is going to need to be optimized better if I want anyone to actually use the new API. ;)
Title: Re: Sphere v2 API discussion
Post by: Rhuan on July 09, 2017, 04:22:16 am
I'm confused by the lack of output for modes 0 and 1 - will recheck later and see if there's a missing line in the copy and paste.

Also can you try with speed limit set to true as well?

For shaders could you have a global variable in the engine saying what shader is in use and only update if it's not the right one? Would have to add the check to all drawing functions but I imagine that it will be quite rare to use multiple shaders so this could result in some speed increase; alternatively maybe make setting a shader a separate action to drawing? So the user has control of it through the the api.
Title: Re: Sphere v2 API discussion
Post by: Rhuan on July 09, 2017, 08:48:33 am
I just pasted the example from above into a new script, set shades to Shader.Default and ran it with modes 0 and 1 and they both drew white rings, so not sure why you got nothing displayed.

Note: Modes 0-2 should draw White rings with the default shader to get a colour output from them need colour masking in your shader but that's not relevant for the speed tests (FPS doesn't change when I use my own shader vs the default).

I've also being thinking about ways to fix the speed issues if they do relate to the points you've identified, what do you think of either:


Adding a "Draw Mode" to the api
Could you wrap the Shader and projection matrices into a "DrawMode" type object that has a default value depending on the sphere version from the manifest (either whatever v1 needs if v1 or an sgm manifest or whatever is needed for the default shader and "standard" v2 drawing is v2) - then enable a function that changes this mode - you could then remove all shader and projection setting from your draw functions. (I note that this would de-couple models and shaders and also mean the script would need a call to extra function if switching between v2 and v1 drawing - but it should drastically reduce the number of graphics card uploads the engine has to do when doing lots of drawing.

Regarding the model view matrices, do they need to be set to null each time?  if shape_draw didn't have its 3rd lineand if group_draw didn't have row 239 would each shape drawn be transformed by the combination of its own matrix and the previous ones or would it be correct (i.e. is screen_transform(g_screen, NULL); purely there to make v1 work?) - if yes it could be dumped into the proposed DrawMode above.

Alternative approach 1 - internal draw mode
Alternatively "DrawMode" could be internally tracked, with a tracker variable of some kind and then a function to do the necessary shader/matrix uploads could be called if an action is being done that's not consistent with the last one.


Alternative approach 2 - separate tracking
Have a v1/v2 mode stored in a boolean in the engine, at the start of each v1 draw function check if currently in v2 mode and if so do the necessary resets, at the end of every draw functions set the boolean to say which mode you're in. -> this should mean that several of the expensive operations are only needed when mixing v1 and v2 drawing.

Similarly you could have a variable tracking what shader is set, using some kind of ID, maybe just an int 0 for v1 shader, 1 for v2 default, 2 for the first custom one loaded 3 for the next etc. Then at the start of each v1 draw function if in v2 mode reset the shader and set the int to 0, if in v1 mode do nothing. At the start of each v2 draw function check if the correct shader is already set per the int, if yes do nothing if no set it.


Uniforms...
I don't see an easy fix for the uniform uploads, at least not without reducing the level of shader control; the valuable features you can put through a shader seem to me to be dependent on setting uniforms specific to whatever you're drawing. Particularly if I want to use them for colour masking and implementing atlases - maybe that's not what they're meant for - but they seem so amazing for that.
Title: Re: Sphere v2 API discussion
Post by: Fat Cerberus on July 09, 2017, 09:19:39 am
Something like your DrawMode idea but implemented internally would probably be good.  I wouldn't want to expose something like that in the API as it's getting a little too close to the state-machine model OpenGL itself uses.  Galileo's stateless object-based model makes it much easier to reason about.

You're right about screen_transform() - by passing NULL it sets an identity matrix for the modelview (and in the next release, an orthographic projection) so that Sphere v1 works.  This is all fallout from Sphere v1 being implemented first - when I wrote the Vanilla implementation, it didn't have to worry about the correct shader/matrix/whatever being set because it was all already set up properly on engine start.  So once I added Galileo, in order for that assumption to still hold and allow v1 and v2 to be combined in the same codebase, Galileo had to be a good citizen and undo all its changes after every draw operation.  Better internal tracking (i.e. decentralizing the responsibility to "undo") would go a long way in speeding things up.
Title: Re: Sphere v2 API discussion
Post by: Rhuan on July 09, 2017, 10:49:45 am

Something like your DrawMode idea but implemented internally would probably be good.  I wouldn't want to expose something like that in the API as it's getting a little too close to the state-machine model OpenGL itself uses.  Galileo's stateless object-based model makes it much easier to reason about.
So something like one of my "Alternate approach" ideas above, possible Alternative Approach 2?

I have been thinking though - will Galileo ever be faster than v1 drawing for simple items assuming we want to be able to move them and recolour them.

Let's compare what TransformBlitMask has to do to what Galileo has to do:

TransformBlitMask has to:
1. Have an image/texture - this will have been uploaded already so nothing to do at draw time.
2. Upload a shape with 4 vertices.
3. Texture the shape with the pre-uploaded texture.
4. Draw.

Galileo has to:
1. Have an image/texture, as above this will have been pre-uploaded.
2. Have a shape with 4 vertices - this will also have been pre-uploaded.
3. Texture the shape with the image.
4. Specify a shader (that's going to do the masking) - following the discussed changes this will hopefully be pre-set
5. Upload a matrix to specify the position.
6. Upload uniforms to specify the colour.
7. Draw

Now assuming the optimisations discussed above are done the minimum actions at draw time will become:

TransformBlitMask:
1. Upload a shape with 4 vertices.
2. Texture the shape with the pre-uploaded texture.
3. Draw.

Galileo:
1. Texture the shape with the image.
2. Upload a matrix to specify the position.
3. Upload uniforms to specify the colour.
4. Draw

So it boils down to whether uploading a shape with 4 vertices is faster or slower than uploading a matrix and some uniforms. (I note that Galileo will likely race ahead for drawing complex objects containing many vertices or models comprised of multiple shapes where the matrix and uniforms can be shared - but frequently a game situation will require objects that move independently of each other so it's hard to say which situation is more common)
Title: Re: Sphere v2 API discussion
Post by: Fat Cerberus on July 09, 2017, 11:23:55 am
Yeah, Approach 2 seems pretty close to the optimization I have in mind.

Regarding that 4-vertex upload: Keep in mind that's 4 vertices per image every single frame.  In Galileo's case, I can optimize away some of the matrix uploads (the projection matrix rarely changes, e.g.), but you're always going to have to pay for those 4 vertices of a transformBlit.

I have a few optimization ideas that should give Galileo a performance advantage even for small workloads, but I'll refrain from mentioning them here until I've had time to test them in-house. :)
Title: Re: Sphere v2 API discussion
Post by: Fat Cerberus on July 09, 2017, 11:36:56 am
See also https://github.com/fatcerberus/minisphere/issues/184#issuecomment-313927121
Title: Re: Sphere v2 API discussion
Post by: Rhuan on July 09, 2017, 11:38:39 am

Regarding that 4-vertex upload: Keep in mind that's 4 vertices per image every single frame.  In Galileo's case, I can optimize away some of the matrix uploads (the projection matrix rarely changes, e.g.), but you're always going to have to pay for those 4 vertices of a transformBlit.

Sure transform Blit has to do 4 vertices per image every frame BUT Galileo has to do a Matrix and some number of uniforms per image every frame (in examples where the images move independently and can change colour), I suppose if we think purely in terms of how many numbers are passed each vertex should be 8 numbers (x, y, u, v, r, g, b, a), so 4 = 32 numbers, whilst a matrix for 2d drawing is 9 and a matrix for 3d drawing is 16. So as long as there aren't too many uniforms the Galileo approach involves less numbers needing to be sent - I'm not sure what extra overheads there are in sending each data type though.


See also https://github.com/fatcerberus/minisphere/issues/184#issuecomment-313927121
Thanks.
Title: Re: Sphere v2 API discussion
Post by: Fat Cerberus on July 09, 2017, 11:56:58 am
An interesting observation I've made is that there is no fixed-function pipeline: Allegro emulates it using a shader.  Even something like setting a matrix amounts to uploading a uniform.  That's why the default shader has uniforms like `al_projview_matrix`.

This is good to keep in mind going forward, as it could have performance ramifications.  All this time I've been assuming the GPU was doing the transformations like in the old days, but it's all handled by the shader now.  That's... rather elegant, actually. :D
Title: Re: Sphere v2 API discussion
Post by: Rhuan on July 09, 2017, 03:08:52 pm
Quick question, did you test the effect of putting the speed limit on - I think there's a serious problem of some kind with the implementation of frame rate limits.
Title: Re: Sphere v2 API discussion
Post by: Fat Cerberus on July 09, 2017, 04:11:33 pm
No, but I can test it out.  If your graphics driver enforces vsync that might be a cause of issues, though.  The framelimiter works by timing frames and then compensates by either adding delays (for short frames) or skipping subsequent frames (if a frame took too long).  If vsync is on the vsync delay can get "out of phase" with the framelimiter and drive the frame rate down even further.  However, that should be self-correcting in the long run.

If vsync is turned off then I have no idea what's wrong.  But I'll give it a go tonight and see if I can find out.
Title: Re: Sphere v2 API discussion
Post by: Fat Cerberus on July 09, 2017, 07:38:33 pm
Okay, you're right, there's definitely something wrong: With vsync disabled in my nVidia settings, I set speed_limit = true and modes 0 and 3 get 60/60 FPS, however modes 1 and 2 get ~30/60 FPS.  This indicates to me that every other frame is being skipped--the engine thinks every frame it draws is late, which doesn't make sense since both modes exceed 60 FPS with the limiter off.  Unfortunately I don't know what's causing it; it's not related to vsync like I initially thought.
Title: Re: Sphere v2 API discussion
Post by: Fat Cerberus on July 10, 2017, 05:31:01 pm
@Rhuan: With my experimental changes to avoid render target changes I managed to get the Galileo mode 0 to run faster (200 FPS) than the Sv1 mode (160 FPS).  Modes 1 and 2 are still slower and I can't figure out why (100 FPS).
Title: Re: Sphere v2 API discussion
Post by: Rhuan on July 10, 2017, 05:49:55 pm

@Rhuan: With my experimental changes to avoid render target changes I managed to get the Galileo mode 0 to run faster (200 FPS) than the Sv1 mode (160 FPS).  Modes 1 and 2 are still slower and I can't figure out why (100 FPS).
I would think that Mode 2 should be the slowest, as it transforms and draws 1,000,000 vertices every frame. (The other 3 modes each draw just 4,000 vertices)

I can't understand why mode 1 would be slow though unless the slow down is from fiddling with the transformation matrix.

Modes 1 and 2 both do the below transformation actions for every ring every frame i.e. 1000 times per frame which Mode 0 doesn't have to do:

Code: [Select]
foo.matrix[1][0] = 3 * (Math.sin(rings[i].t * Math.PI/45) - Math.cos(rings[i].t * Math.PI/45)) / 8;
foo.matrix[1][1] = 0.5 - Math.sin(rings[i].t * Math.PI/45) - Math.cos(rings[i].t * Math.PI/45);
foo.matrix[1][2] = 3 * Math.cos(rings[i].t * Math.PI/45);
//...a few lines later
foo.translate(0, 50 * Math.cos(rings[i].t * Math.PI/100));

Title: Re: Sphere v2 API discussion
Post by: Rhuan on July 10, 2017, 06:07:46 pm
I forgot to say above, great work fixing the rendering speed though from 90 FPS to 200 FPS is brilliant.
Title: Re: Sphere v2 API discussion
Post by: Fat Cerberus on July 10, 2017, 07:17:46 pm
One issue I'm having is that rendering to Sv1 surfaces doesn't seem to always work.  I can't reproduce it in isolation though, it only shows up with the textboxes in Spectacles (which use v1 surfaces), the text doesn't show up.  I'm guessing it's something to do with the transformation matrix not being set or something, but I haven't been able to pin it down yet.  It's always something, isn't it...
Title: Re: Sphere v2 API discussion
Post by: Fat Cerberus on July 10, 2017, 11:48:27 pm
Alright, the surface bug seems to be fixed and I've committed my changes.  Sphere v1 performance remains the same, but Galileo is much faster now and even beats the v1 primitives in performance.  I touched a lot of code to implement the fix, but ultimately it was a simple 3-pronged approach:



I have a feeling there might be a bug with the shader management because Allegro tracks them per-bitmap whereas my code tracks the current shader globally.  But other than that, everything seems to work properly now. :D

edit: As for that test code, I get about 195 FPS for mode 0 when running under SSJ, and ~215 FPS for the non-debug engine.  That's a good sign: It means rendering performance is good enough now that JS execution is starting to become the bottleneck instead.  I still get no output though.  I thought maybe that might be something to do with my graphics driver, so I tested it on my laptop with its Core i7 iGPU and... same thing.  Modes 0 and 1 produce no output, only modes 2 and 3 show any rings.
Title: Re: Sphere v2 API discussion
Post by: Rhuan on July 11, 2017, 04:12:29 am

edit: As for that test code, I get about 195 FPS for mode 0 when running under SSJ, and ~215 FPS for the non-debug engine.  That's a good sign: It means rendering performance is good enough now that JS execution is starting to become the bottleneck instead.  I still get no output though.  I thought maybe that might be something to do with my graphics driver, so I tested it on my laptop with its Core i7 iGPU and... same thing.  Modes 0 and 1 produce no output, only modes 2 and 3 show any rings.
Good news on the speed. The lack of output is really really weird though as they all draw for me.

The common feature of modes 0 and 1 is that they both use a textured shape (mode 2 has no texture)
Title: Re: Sphere v2 API discussion
Post by: Fat Cerberus on July 11, 2017, 10:24:10 am
It seems like something must be going wrong with the texture generation.  I added this code:
Code: [Select]

var ringtex = output.toTexture();
while(true) {
    require('prim').blit(screen, 0, 0, ringtex);
    screen.flip();
}


And nothing shows up on the screen.

edit: It turns out that render-to-texture is broken in my latest builds.  I checked out v4.6.0 and rings show up in all modes.  Then I made a little test program to draw a line loop to a surface and that's broken in the latest build too (it fills the surface instead of drawing lines).  So at least now I have a lead.  More bugs, what fun... :P
Title: Re: Sphere v2 API discussion
Post by: Fat Cerberus on July 11, 2017, 12:47:35 pm
The bug looks like something screwy with the transformations.  The following code fills the surface with red when it should just draw a single pixel in the corner:

Code: (javascript) [Select]

const Prim = require('prim');
var s = new Surface(100, 100);
Prim.drawPoint(s, 0, 0, Color.Red);
var i = s.toTexture();
while (true) {
    Prim.blit(screen, 5, 5, i);
    screen.flip();
}


I haven't been able to figure out what's wrong.  git bisect says the bug was introduced here:
https://github.com/fatcerberus/minisphere/commit/910ad38a131ed85d372b8fb5080d76f034379ffe
but I have no idea what part of that change is causing it.

edit: Dammit, that shader bug bit me after all.  Galileo was using the wrong shader when rendering to a surface because of this:
Quote
I have a feeling there might be a bug with the shader management because Allegro tracks them per-bitmap whereas my code tracks the current shader globally.
Title: Re: Sphere v2 API discussion
Post by: Rhuan on July 11, 2017, 04:20:27 pm

edit: Dammit, that shader bug bit me after all.  Galileo was using the wrong shader when rendering to a surface because of this:
Quote
I have a feeling there might be a bug with the shader management because Allegro tracks them per-bitmap whereas my code tracks the current shader globally.

At least you've found the issue.
Title: Re: Sphere v2 API discussion
Post by: Fat Cerberus on July 11, 2017, 05:56:20 pm
For now I fixed it by just unconditionally setting the shader for every draw operation, but the graphics driver isn't guaranteed to optimize that (i.e. by ignoring a request to bind the same shader), so to avoid dragging performance down, a more proper fix would be to track the active shader per-surface.  On the plus side, switching shaders doesn't seem to be nearly as expensive as setting the render target, likely because doing so involves Allegro creating an FBO for the texture each time.
Title: Re: Sphere v2 API discussion
Post by: Rhuan on July 12, 2017, 03:32:26 am
What did the extra set shader commands do to the speed?

Separate question - if you have Dispatch Update and Render scripts and you want to terminate them without using Sphere.exit() how do you do it? (I tried setting variables to store their tokens then using Dispatch.cancel(token)
Code: [Select]
var tokens =[];
tokens[0] = Dispatch.onRender(drawing);
tokens[1] = Dispatch.onUpdate(updating);


//then within the function "updating" on meeting a ertain condition
Dispatch.cancel(tokens[0]);
Dispatch.cancel(tokens[1]);


But I got a hard crash:

Application Specific Information:
abort() called
*** error for object 0x7fefd985d5d0: pointer being freed was not allocated

Thread 4 Crashed:
0   libsystem_kernel.dylib           0x00007fffac2d9d42 __pthread_kill + 10
1   libsystem_pthread.dylib          0x00007fffac3c7457 pthread_kill + 90
2   libsystem_c.dylib                0x00007fffac23f420 abort + 129
3   libsystem_malloc.dylib           0x00007fffac32efe7 free + 530
4   sphereRun                        0x000000010ab2911c async_cancel + 206 (async.c:67)
5   sphereRun                        0x000000010ab3ceb7 js_Dispatch_cancel + 26 (pegasus.c:1429)
6   sphereRun                        0x000000010ab1c8bf duk__handle_call_inner + 675 (duk_js_call.c:1617)
7   sphereRun                        0x000000010ab1e3c1 duk__js_execute_bytecode_inner + 2373 (duk_js_executor.c:4533)
8   sphereRun                        0x000000010ab1d672 duk_js_execute_bytecode + 150 (duk_js_executor.c:2404)
9   sphereRun                        0x000000010ab1c988 duk__handle_call_inner + 876 (duk_js_call.c:1573)
10  sphereRun                        0x000000010ab48685 script_run + 129 (script.c:177)
11  sphereRun                        0x000000010ab2930f async_run_jobs + 105 (async.c:125)
12  sphereRun                        0x000000010ab480db screen_flip + 1549 (screen.c:383)
13  sphereRun                        0x000000010ab41d31 duk_safe_event_loop + 55 (pegasus.c:900)
14  sphereRun                        0x000000010aae9add duk_handle_safe_call + 610 (duk_js_call.c:2145)
15  sphereRun                        0x000000010aae7fb8 dukrub_safe_call + 34 (duk_rubber.c:64)
16  sphereRun                        0x000000010ab41ce5 pegasus_run + 38 (pegasus.c:696)
17  sphereRun                        0x000000010ab3276a _al_mangled_main + 3177 (main.c:280)
18  sphereRun                        0x000000010ab7db7a +[AllegroAppDelegate app_main:] + 23
19  com.apple.Foundation             0x00007fff98454b3d __NSThread__start__ + 1243
20  libsystem_pthread.dylib          0x00007fffac3c493b _pthread_body + 180
21  libsystem_pthread.dylib          0x00007fffac3c4887 _pthread_start + 286
22  libsystem_pthread.dylib          0x00007fffac3c408d thread_start + 13
Title: Re: Sphere v2 API discussion
Post by: Fat Cerberus on July 12, 2017, 09:59:29 am
It hurt the framerate quite a bit on my GTX 1070 - from ~215 FPS down to around 120.  That's still better than the 87 I was getting before, but not all that great either.

I tried a different hack where the shader is forcibly set right after a render target change and that improved the situation to around 160 FPS.  I'm not sure why I'm not getting the full 215 with that hack - your code is only drawing to the screen during the main frame loop as far as I can tell, so the render target should never change.
Title: Re: Sphere v2 API discussion
Post by: Fat Cerberus on July 12, 2017, 10:04:05 am

Separate question - if you have Dispatch Update and Render scripts and you want to terminate them without using Sphere.exit() how do you do it? (I tried setting variables to store their tokens then using Dispatch.cancel(token)
Code: [Select]
var tokens =[];
tokens[0] = Dispatch.onRender(drawing);
tokens[1] = Dispatch.onUpdate(updating);


//then within the function "updating" on meeting a ertain condition
Dispatch.cancel(tokens[0]);
Dispatch.cancel(tokens[1]);


But I got a hard crash:

Application Specific Information:
abort() called
*** error for object 0x7fefd985d5d0: pointer being freed was not allocated

Thread 4 Crashed:
0   libsystem_kernel.dylib           0x00007fffac2d9d42 __pthread_kill + 10
1   libsystem_pthread.dylib          0x00007fffac3c7457 pthread_kill + 90
2   libsystem_c.dylib                0x00007fffac23f420 abort + 129
3   libsystem_malloc.dylib           0x00007fffac32efe7 free + 530
4   sphereRun                        0x000000010ab2911c async_cancel + 206 (async.c:67)
5   sphereRun                        0x000000010ab3ceb7 js_Dispatch_cancel + 26 (pegasus.c:1429)
6   sphereRun                        0x000000010ab1c8bf duk__handle_call_inner + 675 (duk_js_call.c:1617)
7   sphereRun                        0x000000010ab1e3c1 duk__js_execute_bytecode_inner + 2373 (duk_js_executor.c:4533)
8   sphereRun                        0x000000010ab1d672 duk_js_execute_bytecode + 150 (duk_js_executor.c:2404)
9   sphereRun                        0x000000010ab1c988 duk__handle_call_inner + 876 (duk_js_call.c:1573)
10  sphereRun                        0x000000010ab48685 script_run + 129 (script.c:177)
11  sphereRun                        0x000000010ab2930f async_run_jobs + 105 (async.c:125)
12  sphereRun                        0x000000010ab480db screen_flip + 1549 (screen.c:383)
13  sphereRun                        0x000000010ab41d31 duk_safe_event_loop + 55 (pegasus.c:900)
14  sphereRun                        0x000000010aae9add duk_handle_safe_call + 610 (duk_js_call.c:2145)
15  sphereRun                        0x000000010aae7fb8 dukrub_safe_call + 34 (duk_rubber.c:64)
16  sphereRun                        0x000000010ab41ce5 pegasus_run + 38 (pegasus.c:696)
17  sphereRun                        0x000000010ab3276a _al_mangled_main + 3177 (main.c:280)
18  sphereRun                        0x000000010ab7db7a +[AllegroAppDelegate app_main:] + 23
19  com.apple.Foundation             0x00007fff98454b3d __NSThread__start__ + 1243
20  libsystem_pthread.dylib          0x00007fffac3c493b _pthread_body + 180
21  libsystem_pthread.dylib          0x00007fffac3c4887 _pthread_start + 286
22  libsystem_pthread.dylib          0x00007fffac3c408d thread_start + 13


You're doing the right thing, so that crash is definitely a bug (well, any hard crash is a bug, really, a sandboxed engine should never crash no matter what stupid thing you do).  What jumps out at me is that async_run_jobs() is on the stack; what it looks like is that I didn't account for a job being cancelled in the middle of actually running it and the engine removing it from the queue while it's still executing causes it to blow up.
Title: Re: Sphere v2 API discussion
Post by: Fat Cerberus on July 12, 2017, 12:04:33 pm
I can reproduce the crash using this code:
Code: (javascript) [Select]

var updateToken = Dispatch.onUpdate(blowUp);
var renderToken = Dispatch.onRender(blowUp);
function blowUp()
{
Dispatch.cancel(updateToken);
Dispatch.cancel(renderToken);
}


That should be enough to come up with a fix.  Thanks for reporting it. :D

edit: The crash doesn't happen in a debug build.  What fun...
Title: Re: Sphere v2 API discussion
Post by: Fat Cerberus on July 13, 2017, 12:47:46 pm
I just added a method to the socket object, .connectTo(), that allows you to reuse a socket after it's been closed.  That's convenient not only for reestablishing dropped connections, but also if an online game wants to change servers on the fly or something, it can use the same socket object, which is handy for encapsulation (e.g. if your connection manager doesn't have access to all the places that might be holding onto a reference to the socket, swapping in a brand new object is not an option).
Title: Re: Sphere v2 API discussion
Post by: Rhuan on July 15, 2017, 07:01:15 am
Documentation related point - the v2 API effectively expects a far more intimate knowledge of "normal" JS functions/objects.

With v1 you could basically use the Math object, the various standard data types and Sphere functions. with v2 there's a high chance of finding yourself needed:
- Date
- TextDecoder
- JSON

And maybe others, - I know the v2 api is still WIP (and I've probably contributed to the length of time it will be WIP with my various comments...) but for documentation/tutorial purposes the built in JS objects that are likely to be needed should perhaps be introduced somewhere.

Alternate note, FS.readFile(filename) - could this just return a string instead of an arraybuffer? As a short cut function where there is likely no plan to write the data back out to file having it in array buffer format seems superfluous.

Am I doing something wrong or is this how I'm meant to load a JSON file:
Code: [Select]
var data = JSON.parse(new TextDecoder().decode(FS.readFile(name + ".json")));
Title: Re: Sphere v2 API discussion
Post by: Fat Cerberus on July 15, 2017, 10:46:29 am
To be fair, JSON very quickly became a common thing to find in a Sphere 1.x developer's toolbox.  json2.js was one of most popular Sphere scripts back in the day. :)

It's true that more knowledge of JS built-ins is needed, but making the v2 API more idiomatic was indeed one of my goals.  The C-like setup of the Sphere v1 API is (debatably) more beginner-friendly, but at the cost of increasing the barrier to entry for more experienced JS developers.  On the other hand, if you're like me and get introduced to JavaScript through Sphere, learning it through Sphere v1 hides too much complexity to be an effective teacher.  For example I didn't understand closures for many years because I took the v1 API at face value.

As for FS.readFile() - I guess I could make it return a string.  The reason for returning a buffer was to allow using the function with binary data, but as shown by the DataReader class, it makes much more sense in that situation to just use a full FileStream.  I can say from experience that any time I've ever used FS.readFile it's always been paired with a TextDecoder step.

I don't understand the reason to need to use Date though, at least any more than you'd use it in Sphere v1?
Title: Re: Sphere v2 API discussion
Post by: Rhuan on July 15, 2017, 11:08:29 am

To be fair, JSON very quickly became a common thing to find in a Sphere 1.x developer's toolbox.  json2.js was one of most popular Sphere scripts back in the day. :)

It's true that more knowledge of JS built-ins is needed, but making the v2 API more idiomatic was indeed one of my goals.  The C-like setup of the Sphere v1 API is (debatably) more beginner-friendly, but at the cost of increasing the barrier to entry for more experienced JS developers.  On the other hand, if you're like me and get introduced to JavaScript through Sphere, learning it through Sphere v1 hides too much complexity to be an effective teacher.  For example I didn't understand closures for many years because I took the v1 API at face value.

As for FS.readFile() - I guess I could make it return a string.  The reason for returning a buffer was to allow using the function with binary data, but as shown by the DataReader class, it makes much more sense in that situation to just use a full FileStream.  I can say from experience that any time I've ever used FS.readFile it's always been paired with a TextDecoder step.

I don't understand the reason to need to use Date though, at least any more than you'd use it in Sphere v1?
Needing Date is for when you want to lock movement speeds or other actions to set time lengths rather than locking them to the frame rate. (as v1 had GetTime but v2 only has Sphere.now which gives you frame counts).

I'm not really saying I want to change the style of the api just saying that there are points around this that should be reflected in documentation or tutorials.
Title: Re: Sphere v2 API discussion
Post by: Fat Cerberus on July 15, 2017, 11:25:50 am

Needing Date is for when you want to lock movement speeds or other actions to set time lengths rather than locking them to the frame rate. (as v1 had GetTime but v2 only has Sphere.now which gives you frame counts).

I'm not really saying I want to change the style of the api just saying that there are points around this that should be reflected in documentation or tutorials.


Yeah, switching to frame counts was a deliberate design choice.  I've done a lot of experiments in the past with time-based updates, and they do work well for getting framerate independence.  The problem with them arises when your game starts to lag; if the framerate drops low enough to actually slow you down (i.e. frameskip is not enough), you're still updating entities at the same rate which can lead to all kinds of issues: Entities getting stuck in walls, teleporting, FP rounding errors building up over time... just all sorts of things that cause glitchy frustration for the player and eventually require workarounds to fix.  So you can sync your game to Date.now() if you know what you're doing, but it's a bit of an advanced technique so I like that there's a learning curve in that particular case.

But yeah, I don't disagree about the documentation.  I thought maybe it was a waste of time documenting built-in objects that are much better documented elsewhere (e.g. MDN), but it might be good to document the JSON functions for example, since they're very useful in save systems, among others.  Plus the manifest format is JSON.
Title: Re: Sphere v2 API discussion
Post by: Fat Cerberus on July 15, 2017, 11:30:43 am
Okay, FS.readFile will return a string in 4.7.  FS.writeFile accepts one, as well.
https://github.com/fatcerberus/minisphere/commit/dd8ecefac4b1e24ab78120a38d5ebf4863890a18
Title: Re: Sphere v2 API discussion
Post by: Fat Cerberus on July 16, 2017, 02:42:50 am

Separate question - if you have Dispatch Update and Render scripts and you want to terminate them without using Sphere.exit() how do you do it? (I tried setting variables to store their tokens then using Dispatch.cancel(token)
Code: [Select]
var tokens =[];
tokens[0] = Dispatch.onRender(drawing);
tokens[1] = Dispatch.onUpdate(updating);


//then within the function "updating" on meeting a ertain condition
Dispatch.cancel(tokens[0]);
Dispatch.cancel(tokens[1]);


But I got a hard crash:
<snip>


This bug is now fixed, along with several other glitches related to the same root cause.  Cancellation should work properly now, even when called from a running job.

Only a few remaining things to implement for miniSphere 4.7, most importantly fully v1-compatible spriteset handling:
https://github.com/fatcerberus/minisphere/milestone/38

Changes so far:
https://github.com/fatcerberus/minisphere/blob/master/CHANGELOG.md

Not bad for 11 days' work!
Title: Re: Sphere v2 API discussion
Post by: Fat Cerberus on July 20, 2017, 01:55:01 pm
@Rhuan: I was able to get your ring demo to top out at ~160 FPS in mode 0.  At one point I had it over 220, but that was only with the shader-switching bug intact (and therefore no output).  The speed appears to be limited by the fact that, when rendering using Galileo, the Sphere backbuffer necessarily uses a different shader from the real GL backbuffer.  So switching render targets on every flip also changes the shader and causes slowdown.  This problem doesn't exist for pure v1 code, as in that case both surfaces will always be using the default Allegro shader and no switching takes place.

Setting the GL backbuffer to always use the same shader as the Sphere backbuffer won't work either, for obvious reasons.

All that said, Galileo rendering is still a lot faster than it was in 4.6.0, and when dealing with a large number of vertices Galileo is definitely the clear winner.

edit: Experimented with different numbers of rings, and with 10k rings both v1 and v2 are about equal (which is to say, abysmal), while with only 100 rings Galileo pulls ahead by a pretty wide margin (900+ FPS vs 770 on my laptop).  So in the end I'm pretty happy with the performance, even if I can't quite reach that nice 215 fps with 1k rings I had before.  Super high framerate doesn't mean much if you can't actually see anything :)
Title: Re: Sphere v2 API discussion
Post by: Rhuan on July 20, 2017, 02:55:30 pm
Hmm would be nice if there was a way to avoid the shader switching but I guess we at least have to switch for the back buffer.

I'm really looking forward to seeing this new improved speed though, sound like great work.
Title: Re: Sphere v2 API discussion
Post by: Fat Cerberus on July 26, 2017, 01:13:48 pm
I'm renaming some APIs to make them more newbie-friendly and self-documenting:

FS.exists() --> FS.fileExists()
FS.resolve() --> FS.fullPath()
FileStream::size --> FileStream::fileSize

Any other functions/properties that anyone thinks could be renamed to make them more approachable?
Title: Re: Sphere v2 API discussion
Post by: Eggbertx on July 26, 2017, 03:11:12 pm
I can understand fullPath, but aren't the first and third functions already self-explanatory?
Title: Re: Sphere v2 API discussion
Post by: Fat Cerberus on July 26, 2017, 03:16:31 pm
FS.exists() wasn't clear whether it worked with directories or not (spoiler: it doesn't).  As for fileSize, I changed the name to make it very clear that it gets the size of the underlying file and not the stream itself.  "Size of a stream" is an ambiguous concept and could refer to, for example, the size of an internal buffer.  I want the API to be self-documenting so ambiguity = bad :)
Title: Re: Sphere v2 API discussion
Post by: Eggbertx on July 26, 2017, 03:52:10 pm
Why not have it detect if it detects a directory as well? And fileSize definitely makes sense, I didn't consider that.
Title: Re: Sphere v2 API discussion
Post by: Fat Cerberus on July 26, 2017, 04:34:37 pm
Sure, but it might be better to have a separate function for directories.  If your code is asking "does this file exist?" to avoid an error and the engine says "yes, it does" then you probably expect it to actually be a file and not a folder.

File systems conflating files with directories at the API level was always something that annoyed me.  In practice you almost never want to treat them interchangeably so the low-level file system functions tend to work against you in that regard. :'(
Title: Re: Sphere v2 API discussion
Post by: Fat Cerberus on July 27, 2017, 02:00:17 pm
I'm adding support for relative paths in FS.fullPath().  This is different than just prepending a base directory, as it will honor prefixes.  To illustrate, suppose you had a function to load map files and it did:
Code: (javascript) [Select]
path = "maps/" + mapName;

If the caller passed a mapName as, e.g. "@/otherMaps/map.xml", this would fail because it would end up with something like "maps/@/otherMaps/map.xml" and won't be able to find the file.  So instead, now you'll be able to do:
Code: (javascript) [Select]
path = FS.fullPath(mapName, "maps");
and that will work correctly with prefixed paths.
Title: Re: Sphere v2 API discussion
Post by: Fat Cerberus on July 31, 2017, 12:16:55 pm
I think I did something right with the Sphere v2 API: I'm teaching my cousin with minimal programming background how to use JS and miniSphere and he's picking up a lot of Sphere v2 things very quickly.  So it looks like I got the newbie-friendliness down, at least. :D
Title: Re: Sphere v2 API discussion
Post by: DaVince on August 01, 2017, 09:19:21 am
That's very encouraging! Also, does this mean that eventually we're going to have another member on here? :P
Title: Re: Sphere v2 API discussion
Post by: Fat Cerberus on August 03, 2017, 12:42:27 am
I added a new SphereFS prefix, $/, which is shorthand for the directory containing the main script.  This makes it easier to do absolute imports/requires even when your main script is in a subdirectory, e.g.

Code: (JavaScript) [Select]
import { DayNightEngine } from '$/inGameClock`;

In Cell, $/ is the root of the source tree, i.e. the directory containing Cellscript.mjs.
Title: Re: Sphere v2 API discussion
Post by: Fat Cerberus on August 09, 2017, 04:01:03 pm
I refactored a bunch more stuff, in particular the file system stuff which still had clear signs of how things were handled before SphereFS was introduced.

I also went into various old source files from early in the engine's development and modernized them, updating the naming conventions, code policy, etc.

Overall, the code is much, much cleaner now.
Title: Re: Sphere v2 API discussion
Post by: Fat Cerberus on August 09, 2017, 10:19:12 pm
Oops, meant to post that in the miniSphere thread.

Anyway, besides the $/ prefix, I also added a FS.directoryExists() function.  It's not as useful as fileExists since FS.createDirectory() will automatically create parent directories as needed, but it might be useful in some cases to know if some directory exists without actually needing to create it on the spot.
Title: Re: Sphere v2 API discussion
Post by: Fat Cerberus on August 10, 2017, 11:45:39 am
New API coming in miniSphere 4.8: the DirectoryStream class!  It allows you to read filenames from a directory one at a time.  It even works with from(), e.g. this query to load tests in the Specs test harness:

Code: (JavaScript) [Select]
let fileNames = from(new DirectoryStream('$/testCases'))
.where(it => !it.isDirectory)
.where(it => it.fileName.endsWith('.js'))
.select(it => it.fullPath);

Much nicer than what we had before, GetFileList().
Title: Re: Sphere v2 API discussion
Post by: DaVince on August 10, 2017, 03:48:39 pm
Much nicer, I would say. There was so much lacking/overly convoluted in the Sphere 1 functions it was a real hassle to work with.
Title: Re: Sphere v2 API discussion
Post by: Fat Cerberus on August 10, 2017, 08:16:34 pm
This came about because I happened to notice in Specs a call to GetFileList() and I realized there wasn't a Sphere v2 equivalent.  I considered just adding FS.listDirectory() or something, but then realized it would be much more extensible (and easier to optimize) as an object similar to FileStream.
Title: Re: Sphere v2 API discussion
Post by: Fat Cerberus on August 11, 2017, 01:37:02 am
Decided to take advantage of the revamped FS.fullPath() function to bring back some of the good things about Sphere v1.  SphereFS is improved in miniSphere 4.8 to support relative paths:

Code: (JS) [Select]
var path1 = FS.fullPath('pig.png', '@/images');  // relative
var path2 = FS.fullPath('@/otherDir/pig.png', '@/images');  // absolute
// path1 = '@/images/whatever.png'
// path2 = '@/otherDir/pig.png'

The system modules can take advantage of that to allow conveniences like:
Code: (JS) [Select]
var pigImage = new Image('pig');  // relative to @/images
var superFatPig = new Image('@/otherDir/pig.png');  // absolute

The low-level Core API of course will still use full paths.
Title: Re: Sphere v2 API discussion
Post by: Fat Cerberus on August 16, 2017, 01:30:58 pm
Now that they're no longer buggy as of mS 4.8.2, I'd like to introduce everyone to the SoundStream object!  You can use this to feed raw audio data directly to a mixer.  For example, using the Aurora.js library I can play mp3 files in JS!

Code: (JS) [Select]
window = global;
RequireScript('@/lib/aurora.js');
RequireScript('@/lib/mp3.js');

let fs = new FileStream('@/music/chartreuseRewind.mp3', FileOp.Read);
let mp3Data = fs.read(fs.fileSize);
let asset = AV.Asset.fromBuffer(mp3Data);
let stream = null;
asset.get('format', format => {
// note: Aurora.js always decodes to float32
stream = new SoundStream(format.sampleRate, 32,  // 32bit = float32
                         format.channelsPerFrame);
});
while (stream === null && Sphere.run());
stream.play(Mixer.Default);
asset.on('data', buffer => {
stream.write(buffer);
Sphere.sleep(0.005);  // run event pump
});
asset.start();
Title: Re: Sphere v2 API discussion
Post by: DaVince on August 17, 2017, 01:36:34 pm
This is amazing, and opens up the engine for so many possibilities when it comes to audio capability. :D

Edit: stuff like this could be pretty interesting for inclusion. https://github.com/mohayonao/timbre.js/
Edit 2: or this for MIDI + soundfont support, with plenty of modification I'm sure. https://github.com/skratchdot/timbre.soundfont.js/
Title: Re: Sphere v2 API discussion
Post by: Rhuan on August 17, 2017, 07:12:35 pm
Could you explain briefly how the project matrices work? Any attempt to use them gives me a blank screen.

(I'm trying to find a way to implement a mode7 equivalent and thought projection may be the obvious answer)
Title: Re: Sphere v2 API discussion
Post by: Fat Cerberus on August 17, 2017, 09:53:40 pm
There's a bit of theory involved there that I feel is necessary to know in order to use them effectively, but I'll try to keep it short.

tl;dr: Starting with an identity matrix for the projection will not get you the expected results.

It's worth noting to start with that the default value of .transform for a surface is NOT an identity matrix!  The default projection is an orthographic transformation matching the surface's resolution, in other words:

Code: [Select]
// w, h = surface size in pixels
defaultProjection = new Transform()
    .project2D(0, 0, w, h, -1, +1);

As far as the GPU is concerned, the render target goes from (-1,-1) to (+1,+1), regardless of its actual resolution (this is called NDC, normalized device coordinates).  So if your projection is identity, that's the box you have to draw within to get anything seen.  The projection matrix tells the GPU how to get from world space ("where things are drawn") to NDC.

2D is easy: You pass to .project2D() the corners of a rectangle in worldspace that you want stretched over the screen.  To prevent distortion, the aspect ratio of the ortho-box should match that of the surface.

For 3D the viewable space is actually a pyramid (technically a frustum), since you get more in view the further away you are.  For that you pass to .project3D() a field-of-view angle, an aspect ratio, and the Z-coordinates of the near- and farplanes, both of which must be nonzero and positive.  Explaining any more here is out of scope of this post since you wanted me to keep it brief. ;)
Title: Re: Sphere v2 API discussion
Post by: Rhuan on August 18, 2017, 02:06:22 am
Thanks for the brief intro, I'm afraid I can't see what I'm missing though :(

Any chance you could make a demo that shows project3D working?

Anything I do with it just gives me a blank screen.
Title: Re: Sphere v2 API discussion
Post by: Fat Cerberus on August 18, 2017, 10:05:15 am
It seems there's a bug where setting screen.transform doesn't get picked up right away.  You can work around it for now by drawing something to a surface right after setting the transformation, then the next time you draw to the screen the new matrix will get picked up.

Anyway, this might help illustrate how everything comes together, we'll rotate the entire screen and then project it into 3D:
Code: (javascript) [Select]
var wh = screen.width / 2;
var hh = screen.height / 2;
screen.transform = new Transform()
    .translate(-wh, -hh)
    .scale(1 / wh, 1 / wh)
    .rotate(0.2, 0.0, 1.0, 0.0)
    .translate(0, 0, -1.0)
    .project3D(90, wh / hh, 0.1, 2.0);

    // workaround for miniSphere bug
    new Shape(ShapeType.Points, new VertexList([{}])).draw(new Surface(1, 1));


Fun thing about the Z coordinate: while the near- and farplanes are specified as positive values, Z coordinates of things you draw should actually be negative!  0 is exactly at the camera (and will be clipped, because you can't have a nearplane at 0), and moves in the negative direction "away" from the screen.  That caught me off-guard for a bit when I first starting playing around with 3D.
Title: Re: Sphere v2 API discussion
Post by: Fat Cerberus on August 18, 2017, 02:52:59 pm
Fun thing about the Z coordinate: while the near- and farplanes are specified as positive values, Z coordinates of things you draw should actually be negative!  0 is exactly at the camera (and will be clipped, because you can't have a nearplane at 0), and moves in the negative direction "away" from the screen.  That caught me off-guard for a bit when I first starting playing around with 3D.

I suspect this is the cause of your black screen - since the nearplane can't be at zero (if you wonder why that is: mathematically it's the same thing as dividing by zero), anything drawn with z = 0 will be clipped out of existence when rendered (this is also why walls tend to disappear if you get too close to them in 3D games).
Title: Re: Sphere v2 API discussion
Post by: Fat Cerberus on August 19, 2017, 12:39:39 pm
To avoid any unpleasant surprises: Starting in mS 4.8.3, transf.rotate() takes an angle in degrees instead of radians.  It's not as mathematically "pure", but much easier for novices to understand.
Title: Re: Sphere v2 API discussion
Post by: Eggbertx on August 19, 2017, 01:44:51 pm
What is the purpose of using radians over degrees?
Title: Re: Sphere v2 API discussion
Post by: Fat Cerberus on August 19, 2017, 01:52:30 pm
What is the purpose of using radians over degrees?

Mathematicians like it?  I'm not sure really, they're awkward to use because 360 degrees is ~6.28 radians (i.e. 2*pi).  Apparently some trig identities work better with radians than they do with other angle measurements, but other than that I can't see any real advantage.  Especially for game development.
Title: Re: Sphere v2 API discussion
Post by: Rhuan on August 19, 2017, 01:56:17 pm
What is the purpose of using radians over degrees?
They make a lot of mathematics simpler - without them you end up having to multiple by pi/180 all over the place if doing any funky trigonometry or geometry.

They actually make a lot more sense than degrees it's just everyone is used to degrees.

As one simple example, draw two radii in a circle, measure the angle between them in radians and multiply by the length of one of them and you get the length of the arc between them (i.e. the distance around the circle's circumference between the two radii).
Title: Re: Sphere v2 API discussion
Post by: Fat Cerberus on August 20, 2017, 02:24:35 am
Debating whether I should add a SphereFS prefix for temp files or not.  The prefix would be %/ and to maintain sandboxing purity, would be guaranteed by the engine to have nothing in it on startup.  Anything written into %/ would be deleted on shutdown.

The main thing that's stopping me is that I'm not sure how useful the ability to write temp files would be in practice, particularly for game development.  Any insights for me to help me decide?
Title: Re: Sphere v2 API discussion
Post by: Rhuan on August 20, 2017, 03:47:34 am
Debating whether I should add a SphereFS prefix for temp files or not.  The prefix would be %/ and to maintain sandboxing purity, would be guaranteed by the engine to have nothing in it on startup.  Anything written into %/ would be deleted on shutdown.

The main thing that's stopping me is that I'm not sure how useful the ability to write temp files would be in practice, particularly for game development.  Any insights for me to help me decide?
The time I've wanted to make temp files has been to enable storing images or sound files alongside other data as sphere could only load these as individual files and couldn't convert bytearrays into images/sound the only way to unpack a document containing such things was to read the binary data for each image/sound then write it out to an individual file and load it.
Title: Re: Sphere v2 API discussion
Post by: Rhuan on August 21, 2017, 04:17:11 pm
A few questions on dataReader.js (I thought I'd use it for loading .rmp files rather than re-inventing the wheel again)

1. I think it's meant to have:
const from = require('from');
at the top, the latest version doesn't have this but it uses from in the _checkStructDescriptor function.

2. Did you remove the ability to have multi-tier objects/arrays? I was looking back around page 74 of the miniSphere development topic and you'd been using a somewhat complex object with branches and self references in it to read an .rmp file, I assumed that this was with an early version of dataReader.js but I can't see any facility for processing of self references or branches in the latest version, am I missing something? If the facility is gone/not returning that's fine I'll write my own just don't want to do it if it's already there somewhere.

Title: Re: Sphere v2 API discussion
Post by: Fat Cerberus on August 21, 2017, 04:56:06 pm
Thanks for the heads up, I'll fix the bug in 4.8.4.

As for the branches and backrefs, I removed them because they were difficult to document adequately.  So I just tried to make it work similarly to a C struct because that's easy to understand and explain.  If there's enough demand I could bring the feature back, though. ;)

I admit I was nice being able to load an entire .rmp file with one call...
Title: Re: Sphere v2 API discussion
Post by: Rhuan on August 21, 2017, 04:57:58 pm
Thanks for the heads up, I'll fix the bug in 4.8.4.

As for the branches and backrefs, I removed them because they were difficult to document adequately.  So I just tried to make it work similarly to a C struct because that's easy to understand and explain.  If there's enough demand I could bring the feature back, though. ;)

I admit I was nice being able to load an entire .rmp file with one call...
I've just learnt how the history feature works on github. :P

I'm going to add the branches/backrefs code to my own script for now, it's too good not to use.
Title: Re: Sphere v2 API discussion
Post by: Fat Cerberus on August 22, 2017, 12:48:30 am
I've just learnt how the history feature works on github. :P

Version control is awesome isn't it? :D  I like using Git because I'm not tempted to keep dead/commented-out code around forever, I can prune things as aggressively as I want and if it turns out I need the code back later I can just get it back from the history.  Sometimes it's difficult to find things, but I've found YAGNI holds true most of the time and I indeed never need the deleted code again.
Title: Re: Sphere v2 API discussion
Post by: Rhuan on August 22, 2017, 06:10:43 pm
Am I allowed to be annoyed that neither textures nor surfaces are readable in sphere v2?
I was briefly tempted to resort to some v1 functions....

(I want to implement fancy pixel scaling algorithms, but I don't want them at render time I want to do them in advance for a stack of reasons)

BUT it's fine I've worked out a work around that may even work better than what I was planning before, it's just not what people would think of. (but it will be cool)
Title: Re: Sphere v2 API discussion
Post by: Fat Cerberus on August 22, 2017, 06:35:28 pm
I actually wanted to add a function for that like Surface#lock() but I couldn't come up with a way to expose the raw pixel data to JS in a user-friendly way.  I'll revisit it at some point, it might be possible to have it return an ArrayBuffer or similar.

That said, the preferred way to do fancy scaling nowadays is in the fragment shader.  I think I've seen hq2x shaders out there, for instance.
Title: Re: Sphere v2 API discussion
Post by: Rhuan on August 22, 2017, 06:40:13 pm
I actually wanted to add a function for that like Surface#lock() but I couldn't come up with a way to expose the raw pixel data to JS in a user-friendly way.  I'll revisit it at some point, it might be possible to have it return an ArrayBuffer or similar.

That said, the preferred way to do fancy scaling nowadays is in the fragment shader.  I think I've seen hq2x shaders out there, for instance.
Yeah what I'm going to do is use shaders to scale and draw to a surface then convert the surface to a texture.

(I want these done in advance not at runtime as my runtime shader is complex enough as it is, though I may revisit that later. Having the whole map as an atlas drawn using a shape the size of the screen with just 4 vertices seemed like such a good idea when I first thought of it, but getting the coordinates and stuff to work properly with it was... interesting to say the least.
Title: Re: Sphere v2 API discussion
Post by: Rhuan on August 22, 2017, 06:52:44 pm
I actually wanted to add a function for that like Surface#lock() but I couldn't come up with a way to expose the raw pixel data to JS in a user-friendly way.  I'll revisit it at some point, it might be possible to have it return an ArrayBuffer or similar.

That said, the preferred way to do fancy scaling nowadays is in the fragment shader.  I think I've seen hq2x shaders out there, for instance.
TBH I''m not sure if there's a need for it as you can do whatever you need with a shader, it just feels counter intuitive to use a shader to fiddle with an image then store the edited image into a variable for later use.

If you were to implement it, I'd want it as a direct inverse of the new Texture function except as a surface method probably.