neoSphere 5.10.1

Topic: neoSphere 5.10.1 (Read 8515179 times) previous topic - next topic

0 Members and 106 Guests are viewing this topic.

Re: minisphere 1.1b4 (stable: 1.0.10)

Reply #465 – May 05, 2015, 11:49:37 pm

v1.1b4 is out, boasting more bug fixes, massively improved performance, OpenGL rendering on Windows and one of my famous subtle-but-awesome changes:

RequireSystemScript() now searches <game_dir>/scripts/lib for scripts first, before the system scripts directory.

Re: minisphere 1.1b4 (stable: 1.0.10)

Reply #466 – May 06, 2015, 01:17:00 am

So I did a stress test in Sphere 1.5, SSFML 0.9.0, and minisphere 1.1b4

This is running at 26fps with 500 entities in SSFML
https://www.dropbox.com/s/cnaylep3xytgqdl/stress-ssfml.png?dl=0

This is running at 21fps with 250 entities in Minisphere
https://www.dropbox.com/s/8p9u3kzejhu8nvv/stress.png?dl=0

This is running at 17fps with 250 entities in Sphere 1.5 in SphereGL
https://www.dropbox.com/s/4nflt2g7cg4mkgc/stress-sphere15.png?dl=0

All fps's are +/- 5.

And well... Sphere 1.5 couldn't do 50 on Standard32. This test was on Intel Core i5 3.3@3.6ghz, with an Nvidia GTX970, 8GB 1066@800Mhz ram. I think it's the JS logic that's bottlenecking the two good engines, and the graphics plus the logic for Sphere 1.5. In SphereGL I got by no different than minisphere, which is telling. You are not taking advantage of the hardware on the computer, it tells me you're still mostly drawing in immediate mode which is what kinda plagued SphereGL and Sphere in general. I know this since I know your JS engine is faster in most cases to Sphere 1.5.

My max target for the space game is 300 enemies on screen. But this rest is more of a curiosity. In the game you never see more than 20 anyways, unless there's a giant space battle programmed into it. But I think the stress test is still neat. No engine crashed BTW.

Ok, so... minisphere 1.1b4 loaded blockman much faster! The combat was also nice and smooth.

But it did crash after playing for a while when I swung my sword at an enemy. So there's something there, maybe a get pixel call out of bounds?

Re: minisphere 1.1b4 (stable: 1.0.10)

Reply #467 – May 06, 2015, 01:35:32 am

I could speed up the windowstyle loading too if I atlased them, but then hardware tiling doesn't work. The main loading bottleneck was fonts and spritesets (and maps), which have been speed up a ton not only due to atlased loading in 1.1b4, but also spriteset pooling.

But yeah, most of the drawing is done in immediate mode because for some strange reason, Allegro doesn't always honor transformations when using batched mode. This is needed because I use a transform to scale up the window for 320x240 games. I'd rather take the performance hit than end up with dodgy rendering.

I assume you noticed your windowstyles render properly now? I bit the bullet and compiled Allegro with the 16x16 texture clamping commented out. So far I haven't had any issues, so... *crosses fingers*

As for those crashes, I've seen a few of them myself, but I haven't tracked down the cause yet. Hence why minisphere 1.1 is still in beta.

Re: minisphere 1.1b4 (stable: 1.0.10)

Reply #468 – May 06, 2015, 02:50:10 am

You might want to stay in immediate mode for now. I'm taking a small risk batching everything. I'm meaning to create a small batching API to make it explicit. But I have found through my testing that even implicit batching is decent enough, and really only helps when you have much of the same sprite, like the above where the enemies and particles and bullets were the same 3 textures.

I guess if you stay in the map engine, implicit batching really is not needed since sprite atlasing covers that. The implicit batching I do to speed up the space game only really works when I program knowing that this behavior exists and that I can draw images in a custom engine rather than use spritesets in the map engine. There is a slight speedup drawing names on their own layer rather than in an A/B pattern. But it's minimal.

Re: minisphere 1.1b4 (stable: 1.0.10)

Reply #469 – May 06, 2015, 03:12:25 am

So it didn't make it into 1.1b4, but I just added a console and BGM manager to the mini runtime, both originally coded for Specs.

The console is output-only (for now), but optionally lets you log output to a file, and because it uses minithreads, lets you pull up the console at any time during gameplay. This little tool has been invaluable in debugging Specs glitches, and is generic enough that I thought it'd make a good addition to the runtime.

The BGM manager right now is a bit of a fixer-upper, but the intent is to turn it into a simple stack-based music switcher. You push to change the song, and pop to return to the previous one (perhaps resuming where that one left off). I might also add a "layer 0" as well for field music, which can be changed in place but only plays when there are no other tracks on the BGM stack.

That puts the minisphere runtime at 6 components:
- Core (provides delegates)
- Link
- minithreads
- miniscenes (Scenario)
- miniconsole
- miniBGM

Quite a nice set of tools it's shaping up to be, methinks.

Re: minisphere 1.1b4 (stable: 1.0.10)

Reply #470 – May 06, 2015, 03:45:46 pm

So I got bored and implemented basic command support in miniconsole. It's a very easy-to-parse command format--e.g. battle rsb2 for a test battle in Spectacles. So now anyone that wants to implement a basic console in their game will have one ready to go!

Usage:

Code: (javascript) [Select]

RequireSystemScript('mini/Console.js');

mini.initialize({
    logFile: "consoleLog.txt",
    consoleLines: 10,
    consoleBuffer: 1000
});

mini.Console.register('bear', {
    'munch': DestroyPerson
});

Then the player could pull up the console and type in:
bear munch lauren

And the map person named "lauren" would get eaten by the bear and disappear. How it is parsed is, the first token is the object name (bear) as passed to mini.Console.register(), the second token is the command name (munch), and the remaining tokens are passed as string parameters to the relevant method. The actual call is also wrapped in a try/catch to try to prevent game crashes due to bad console input.

Fixed tt tag ~radnen

Re: minisphere 1.1b4 (stable: 1.0.10)

Reply #471 – May 08, 2015, 03:01:56 am

minisphere 1.1 is going to be delayed by a bit, it seems. I apparently broke something when adapting Scenario and now it locks up sometimes during scene execution. I haven't been able to track down the cause. It's odd, the game seems to completely lock up, but closing the minisphere window works, which means FlipScreen is being called.

I suspect a bad interaction between the Scenario threader and minithreads is causing a deadlock. I didn't even think that was possible with cooperative threading (making a distinction between a deadlock and an infinite loop, before anyone calls me on it), but...

Re: minisphere 1.1b4 (stable: 1.0.10)

Reply #472 – May 09, 2015, 01:53:20 am

So I think I'm going to end up releasing minisphere 1.1 without the runtime.

As it stands right now, all runtime components rely heavily on threads. And as I've just found out, minithreads has a rather massive design flaw: Because join is emulated with recursion, joins don't always do what you expect: Assuming two simultaneous joins, satisfying the first one is--by definition of recursion--necessarily contingent on the second one returning. Even if the first join would otherwise be satisfied first, in practice it can't be because one further down the rabbit hole is still blocked. Sometimes--aw, who am I kidding, way too often--this can even cause a deadlock.

For whatever the reason, as long as I've been using this threader in Specs, this never bit me before now. I guess Specs alone just didn't tax the system enough to flush it out--it wasn't until I tried to reimplement Scenario in terms of joins that it showed up, and struck HARD as Scenario absolutely requires reliable concurrency. What sucks is that it took me all day to figure it out, after many exhausting hours of misguided hacking trying to fix what couldn't be fixed.

Curious how I finally figured it out? I added two GetSeconds() calls to Scenario's tween scenelet, one in start and another in finish. And lo and behold, one particular tween that was supposed to take half a second... didn't finish up until 12 seconds later. I puzzled for a bit, and then it dawned on me what was going on... and I felt like a complete idiot.

Re: minisphere 1.1b4 (stable: 1.0.10)

Reply #473 – May 09, 2015, 02:04:52 am

Quote from: Lord English – on May 09, 2015, 01:53:20 am

Curious how I finally figured it out? I added two GetSeconds() calls to Scenario's tween scenelet, one in start and another in finish. And lo and behold, one particular tween that was supposed to take half a second... didn't finish up until 12 seconds later. I puzzled for a bit, and then it dawned on me what was going on... and I felt like a complete idiot.

That's not being an idiot, sometimes sanity checks once and a while help. Now, how are you going to solve it, that's the question. Deadlocks usually arrive because of a lack of concurrency handling, every threaded system has mutex locks, semaphores even busy-wait loops, you just need to pick solution(s) that work.

But I don't think this is a deadlock. It might be a race condition since after 12 seconds it finished rather than .5... a deadlock is most usually... dead! A race condition means things are vying for cycles and other loops/threads get pushed back. In fact I'm sure that 12 seconds is variable. Is it 13 next time, immidiate another, 5 seconds?

Re: minisphere 1.1b4 (stable: 1.0.10)

Reply #474 – May 09, 2015, 02:24:51 am

Me being an idiot wasn't referring to the sanity check, it was that, knowing how recursion works, this eventuality never even crossed my mind.

But the 12-second tween indeed wasn't a deadlock. However, deadlocks most certainly can (and DO) happen with this setup.

Here's the implementation of join():

Code: (javascript) [Select]

mini.Threads.join = function(threadIDs)
{
	threadIDs = threadIDs instanceof Array ? threadIDs : [ threadIDs ];
	
	if (!this.isInitialized)
		Abort("mini.Threads.join(): must call mini.initialize() first", -1);
	var isFinished = false;
	while (!isFinished) {
		this.doFrame();
		isFinished = true;
		for (var i = 0; i < threadIDs.length; ++i) {
			isFinished = isFinished && !this.isRunning(threadIDs[i]);
		}
	}
};

Note that it loops until the requested thread(s) finish. This works well enough--if there is only one outstanding join. Trouble is, Scenario is often waiting on many things at once. At this point, the recursive system breaks down.

Let's simplify things a bit and say we have four running threads, call them A, B, C and D.
* A joins B, which prevents A from being updated until B terminates. So far so good.
* .join() is running its own loop, so B, C, and D keep updating.
* At some point (A is still waiting on B to finish), C joins D.
* B terminates. A's join now can't be satisfied because, by definition of recursion, it is itself blocked by C's join. D may not terminate until some way down the line, which means it's unintentionally holding up the works.

And yes, this system can give rise to a deadlock very easily. In the simplest case, this will do it:
* A joins B.
* B joins A.

In practice, such cycles will be (and were!) much more roundabout and therefore difficult to debug.

Note that, due to the way the system is designed, the effects of this may not be immediately obvious as other threads will continue to update (just like preemptive threading!). However, if what A is trying to accomplish is necessary for the game to continue (hence why B wanted to wait on it in the first place), this will put the game in an unrecoverable state.

Re: minisphere 1.1b4 (stable: 1.0.10)

Reply #475 – May 09, 2015, 11:40:15 am

Speaking of sanity checks, I added Assert yesterday. The API signature:

Code: (javascript) [Select]

Assert(condition, errorText[, stack_offset]);

I made the error text mandatory instead of optional to encourage meaningful asserts. If an assert fails (the condition is false), the specified error text will be displayed in a modal popup allowing the user to choose whether to continue or terminate. As with Abort(), the optional stack offset is used to blame the failure on a function earlier in the call stack.

Hm... now that I think about it, Assert should probably behave differently in the console vs. release engines. In a release build (engine.exe, engine64.exe), a failed Assert should terminate the engine unconditionally (without giving you the yes/no options), while the Console build (console.exe, console64.exe) would log the assert to stderr and give you the option to try to continue.

As for that threader issue, what I think I'm going to do is modify the semantics of join. Rather than block the caller directly, it will store the joined tids in the calling thread's "join list", and that thread will not get any further update() calls until all the joined threads have terminated. This will require a few architectural changes to Specs, but so be it. I'll definitely miss the blocking .join() though. It almost perfectly emulates Pthreads join, but that "almost" turned out to be the killer.

Re: minisphere 1.1b4 (stable: 1.0.10)

Reply #476 – May 09, 2015, 05:26:21 pm

I have found out that in our engines, SSFML and minisphere, this causes a considerable speedup to getPixel:

Code: (javascript) [Select]


	function SurfaceWrap(image, hold) {
		this.surface = image.createSurface();
		this.width = image.width;
		this.height = image.height;
		
		var pixels = [];
		
		this.update = function() {
			var w = this.width, h = this.height;
			for (var x = 0; x < w; ++x) {
				pixels[x] = [];
				for (var y = 0; y < h; ++y) {
					pixels[x][y] = this.surface.getPixel(x, y);
				}
			}
		}
		
		if (!hold) this.update();
		
		this.getPixel = function(x, y) {
			return pixels[Math.floor(x)][Math.floor(y)];
		}
		
		this.rotate = function(r, s) {
			this.surface.rotate(r, s);
			this.update();
		}

		//... other methods...
	}

I think it has to do with the domain bounds, between script engine and native execution because even with native side pixel caching, engine based caching is faster to retrieve from. The only downside to the above is, notice the rotate, I have to call the update method each time to get a fresh list. But my code only rotated the image once. The speedup isn't huge, though. Testing on 500 static images, the engine used to halt entirely in our engines. With this addition there is no visible halting, everything is just slow, like 10-17fps slow, but consistent which says this is much faster. Though in practice it may be slower, especially if you modify the surface a lot with rotations and other methods (I only wrapped the rotate method).

In Sphere1.5 the above errors with an out of memory error, after about 40 entities.

Re: minisphere 1.1b4 (stable: 1.0.10)

Reply #477 – May 09, 2015, 07:21:02 pm

This makes sense. When calling getPixel() directly, even though the engine is caching it, you still have the overhead of a function call per pixel. In Duktape, a JS->C call is identical to a JS-JS call except for the last step (call into native instead of bytecode)--in fact it may even be faster because some of the call setup can be skipped (arguments object creation, etc.). But the fact remains that calling a function in JS is a lot more expensive than in native. Caching the pixels locally avoids that, as then the JS engine just has to retrieve values from an array, which is trivial.

By the way, the out of memory errors in 1.5... You will see those a lot. It only allocates 5MB to the JS context!

Re: minisphere 1.1b4 (stable: 1.0.10)

Reply #478 – May 11, 2015, 03:35:30 pm

I just added a new API: DoAsync(). This queues a script to be executed on the next FlipScreen. It's like what setTimeout(fn, 0) does in a browser. I added this mostly because of minipact, since according to the Promises/A+ spec, promise resolution must be asynchronous, outside the usual program flow.

I'm sure this has other uses, though...

Re: minisphere 1.1b4 (stable: 1.0.10)

Reply #479 – May 12, 2015, 03:48:02 pm

So using promises for minithreads turned out to be inadequate. They work well enough for Scenario which creates a bunch of one-off threads and runs them in parallel, only occasionally stopping to resync timelines. But for more complex threading, like waiting on a menu thread during a battle, things get awkward because the battle thread still updates in the meantime.

What I'm going to do is change the semantics of join(). It will still return a promise, but it will ALSO exclude the calling thread from further update() calls until the threads being waited on finish. This way join() would still work mostly like it does in Pthreads without holding up the works with recursive event loops, and the promise would take care of post-join logic (such as queuing a move, etc.)

Edit: On second thought, I'm not even going to do that. Blocking join works fine for 90% of use cases, it's only interlacing joins that cause issues. I'll just make a note not to do that in the documentation.