So far, there are two overarching goals for the minisphere 5.0 release:

• Cell should actually be useful. First seen in minisphere 2.0, Cell development has basically been at a standstill ever since.  A Cellscript currently serves two purposes: To write out the game manifest, and to select which files make it into the final game package.  Since a Sphere v2 manifest is just a JSON file, it's easy enough to write it manually (and you pretty much do this even with a Cellscript since the manifest() asset just JSON encodes whatever you pass it), and copying files can be done equally well with a shell script.  Thus Cell is really just a glorified SPK packer in its present state.  To make it useful, the plan for 5.0 is to add TypeScript and CoffeeScript transpilation, and maybe minification, and to automate the JSON manifest building so that it's worthwhile to let Cell do it vs. just writing it yourself.
  
  


• Faster command-line development. Even with the enhancements above, sometimes Cell will still be overkill, and this is what the Majin tool mentioned above is working towards.  It should always be possible to create a working Sphere game entirely using the command-line, but it is tedious to edit JSON data by hand, particularly as the number of properties in a file grows.  Majin is meant to make things easier when working with the Sphere v2 JSON manifest format.

Can I ask what the idea behind Cell is? I never really liked the idea of spks  as I liked being able to see all the documents in my games and being able to use custom folder structures within them - maybe I was misunderstanding something but an spk didn't seem to add any value and potential made the above harder.

When I first saw mention of "the cell compiler" I thought this was going to be somehow compiling the JS to create massive optimisations but I assume a) I was wrong AND b) it's never going to do that?

You don't have to make an SPK, Cell can also compile to a loose directory.

The idea is to automate Sphere package creation, whether that "package" is a directory or SPK.  One of the features planned for minisphere 5.0 is TypeScript support, but Duktape is too slow to do that on the fly, so the idea is to use Cell to compile the TypeScript to JS, minify scripts, etc.  In other words it's a build step.  That's completely optional of course - if you want to code directly in JS and in general use the supported formats directly, that's fine too.

Follow-up to the above: The full minisphere toolchain actually consists of a few executables:

• minisphere: The end-user engine.  This is what you want to distribute with your game.  On Windows it is fully static linked and requires no external DLLs, only the system folder.  Console output is disabled, and the debugger is also disabled, improving performance.  In Windows, no console window is shown.


• spherun: The Sphere developer's engine.  This enables console output as well as single-step debugging through either SSJ or Sphere Studio.  It also starts up in windowed mode (the end-user executable starts fullscreen).


• cell: The Sphere v2 compiler.  It doesn't do much in 4.3.8, but as I've mentioned, I'm in the process of a major overhaul for minisphere 5.0.  Its purpose is to automate TypeScript transpilation, minification, and anything else needed to prepare a game for distribution.  The whole build process is controllable using a JS script.  The overall design is inspired by SCons, but it's only for Sphere and uses JavaScript instead of Python.  It also supports incremental builds (only copying/building files that have changed).


• ssj: A JavaScript command-line debugger for Sphere.  It lets you step through your game's JS code, view variables, etc. to find bugs.  This is basically GDB for Sphere.


• majin: Not yet available.  This is a planned utility for minisphere 5.0 that will allow you to edit your game manifest (i.e. game.json) and create SPK packages from the command line.
  

I should probably try and get sphererun compiled on mac next and then do cell and ssj - though I've got sidetracked beginning to script systems for a game now I have sphere working.

(Also I don't know what GDB is - remember until the last week or so it was a couple of years since I'd done any programming unless you count excel formulae... So in general please expect me to not know what lots of common modern programming/scripting tools are)

gdb is the GNU debugger. On Mac, you probably have lldb, the llvm debugger. Both can debug executables from either toolchain. Their syntax is not quite compatible (lldb tends to be more verbose, I far prefer gdb).

So I've been playing around a bit with miniSPhere - it's implementation of the 1.5 api seems to be missing a few of the graphic primitives:

1. FilledCircle - miniSPhere has surface_object.filledCircle but no vanilla Filled Circle (it has GradientCircle which can be used instead but is assumedly slightly slower)

2. Ellipse functions in general the 1.5 api lists quite a few of them both in the primatives section and the surfaces section, I can't see any in miniSphere's vanilla.c

I've probably missed one or more; but hey loops and pixel  operations get me there anyway...

GradientCircle is probably not any slower in reality.  It's hardware accelerated and is drawn as a triangle fan in both the solid and gradient cases, so there's no real difference unlike with software rendering.  That said, the missing functions *are* very much a compatibility issue, so I'll add them.  Thanks for pointing them out.

Found another engine difference between 1.5 and miniSphere. Abort in 1.5 would parse new line instructions (/n) And would not say the exit was an error if /n was included in the text.

(Some games used this as the actual method of exiting the game and giving the user a goodbye message or something - obviously it's not a great method as it's giving them white on a black screen etc. so likely not going to be used in any serious project but hey)

Yeah, I didn't bother emulating that Abort() behavior, it didn't really seem worth it.

I'll make a 4.3.9 release in a day or so to add the missing ellipse functions.  I can't do it immediately because the trunk is in shambles thanks to the ongoing Cell overhaul, so I'll have to make a branch off of 4.3.8 and cherry-pick the relevant fixes.

Another question... Possibly more a feature request.

Something I wanted to be able to do before was make hybrid custom data files so I could store mixed game resources in one document. The problem with this is Sphere does not support converting a byte array to an image or sound or vice versa.

The only way I could this before with images was to write the image to a file then open it as a rawfile, read the data and then write it into my data file - then to load it read the data, write it to a temporary file with an appropriate image extension then use the LoadImage function - there was no version I could do at all with sound.

I had been hoping to use a custom spriteset format in the project I'm working on but it looks like I probably can't as it will involve way too much disk input/output to load it - shouldn't there be a facility to convert a byte array to an image or other equivalent functions without having to write it to disc?

And got another issue for you; this time an error in miniSphere.... Sorry I have a tendency to use obscure functions.

Background: I used an online sprite generator which creates an image output containing the various frames of the sprite; as I'm intending to use it quite a bit AND it also includes some stuff at the bottom which prevents sphere's image to spriteset tool working I wanted to write a script that could read lots of outputs form this sprite generator and make them into sphere spritesets (or some other custom spriteset format though probably the former due to the point in my previous post).

However miniSphere can't do it quite right - Sphere 1.5 running under Wine can.

Method:
1. Load the image as a surface
2. use nested loops to slice the image up surface_object.cloneSection(x,y,w,h).createImage();
3. do stuff with the resultant images

Problems (in each case no problem in 1.5 but a problem in miniSphere):
1. whilst the image loads with its background being transparent initially after it's sliced up the background becomes opaque in miniSphere - Sphere 1.5 keeps it transparent
2. the createImage() function in minisphere somehow fails to clean its buffer - if I remove the createImage step and blit  a few of my outputs in miniSphere they look ok apart from the transparency issue BUT if I put the createImage step back each frame becomes the result of bliitng all previous frames on top of each other seemingly with transparent backgrounds except a non-transparent background is added to the whole thing at the end.

edit: and another point miniSphere doesn't have the CreateSpriteset function.
- so far miniSPhere can run all my game code but not utility code - but sphere 1.5 under wine can run the utility stuff so it's ok

later today I'll try and post a completeness check of the miniSphere functions as I seem to keep finding ones it's missing.

I had been hoping to use a custom spriteset format in the project I'm working on but it looks like I probably can't as it will involve way too much disk input/output to load it - shouldn't there be a facility to convert a byte array to an image or other equivalent functions without having to write it to disc?

With the Sphere v1 API, you're right, there isn't a way to do this.  However, Sphere v2 does include such a thing:
https://github.com/fatcerberus/minisphere/blob/v4.3.8/docs/sphere2-api.txt#L769-L777

To get an ArrayBuffer from a file, use the FileStream API:
https://github.com/fatcerberus/minisphere/blob/v4.3.8/docs/sphere2-api.txt#L1088


For sounds, you could look into the SoundStream API, though it may not be an exact match to what you need:
https://github.com/fatcerberus/minisphere/blob/v4.3.8/docs/sphere2-api.txt#L1047-L1085

Sorry, but you're going to have to learn the new API at some point.

I'll respond to your other post in a bit.

Thanks for the pointers... Not so sure on learning this new api, I know the old one so well that trying to replace it in my mind would be a lot of effort.

As for the other points, no rush - you've done an awesome job making a modern rebuild of sphere that generally runs faster and does stuff better whilst using a lot of code - the face that I can find a few bits that don't work at the moment isn't a massive issue.

And got another issue for you; this time an error in miniSphere.... Sorry I have a tendency to use obscure functions.

That's fine, good to have someone testing the darker corners of the engine.   Keep in mind though: The Sphere v1 API is deprecated.  I'll maintain is so far as fixing compatibility issues and obvious bugs, but I won't go out of my way to accommodate it.  That having been said, moving on...

Background: I used an online sprite generator which creates an image output containing the various frames of the sprite; as I'm intending to use it quite a bit AND it also includes some stuff at the bottom which prevents sphere's image to spriteset tool working I wanted to write a script that could read lots of outputs form this sprite generator and make them into sphere spritesets (or some other custom spriteset format though probably the former due to the point in my previous post).

However miniSphere can't do it quite right - Sphere 1.5 running under Wine can.

Method:
1. Load the image as a surface
2. use nested loops to slice the image up surface_object.cloneSection(x,y,w,h).createImage();
3. do stuff with the resultant images

Problems (in each case no problem in 1.5 but a problem in miniSphere):
1. whilst the image loads with its background being transparent initially after it's sliced up the background becomes opaque in miniSphere - Sphere 1.5 keeps it transparent
2. the createImage() function in minisphere somehow fails to clean its buffer - if I remove the createImage step and blit  a few of my outputs in miniSphere they look ok apart from the transparency issue BUT if I put the createImage step back each frame becomes the result of bliitng all previous frames on top of each other seemingly with transparent backgrounds except a non-transparent background is added to the whole thing at the end.

Your issue with createImage "not clearing its buffer" seems weird.  ssSurface#createImage() does this:
https://github.com/fatcerberus/minisphere/blob/v4.3.8/src/engine/vanilla.c#L4169-L4182

There's no buffering involved - it just clones the contents of the surface (in fact surfaces and images are the same thing to the engine internals, it's only the API that's different).  So not sure what's causing that errant compositing behavior.  Seems weird.

edit: and another point miniSphere doesn't have the CreateSpriteset function.

CreateSpriteset() was left out on purpose, actually.  Best I can tell, how Sphere 1.x implements spritesets is by using the JavaScript spriteset object directly, without having a dedicated internal structure.  The JS engine and map engine are quite tightly coupled.  minisphere tries to decouple them as much as possible and has its own internal spriteset type, so getting the two domains (JS and C) to cooperate is harder.  I might be able to swing something though, I'll see what I can do when I have some spare time.  Thanks again for testing, it's appreciated!