Skip to main content ↑ ↓
Topic: TurboSphere (Read 2437066 times) previous topic - next topic
0 Members and 4 Guests are viewing this topic.

Re: TurboSphere

Reply #480 –
I fixed changing the Image of a Shape.

Trying to use this uncovered a HUGE problem with modifying Images and how they are managed. It's all fixed now, and I'm glad I caught the issue so soon.

Now, Images are properly maintained until their Image object and all Shapes that use the Image are destroyed, and changing the Image of a Shape using

Code: [Select]

some_shape.image = new Image("image.png");

correctly removes the reference for the old image.

Previously code like that would ALWAYS destroy the underlying GL texture of the old image, which would either cause that image to be garbage or blank, or (most perplexingly) the next call to `new Image' would create a new GL texture of the same name. This made the old Images that were replaced hold the new texture, because the old image used the old GL texture name, which was now the new image's texture, and had been overwritten by the new Image!

That took the last two days to figure out. It was fun, but also quite a nice challenge..
 

Re: TurboSphere

Reply #481 –
I just found out that V8 moved repos just a little bit ago (November 2014) on Github without much notice.

The new build system is even more broken. It refuses to directly build with anything but a prebuilt compiler that it insists on downloading (which is almost 400 MB). I finally got all that working, and I find that V8 has lost the ability to be directly compiled into a shared library. Legend has it there is a new switch to enable this, but it does not seem to work.

TurboSphere literally cannot use V8 as a static library.

In addition, running the included tests for V8 show it has actually regressed in performance since the older version I've been pulling, by almost 8%. It seems that TurboFan and CrankShaft are in some kind of weird conflict, particularly on non-jsm scripts.

I wonder how hard it would be to compile SpiderMonkey...

Re: TurboSphere

Reply #482 –
If you do switch to SpiderMonkey, would we get the benefit of being able to use the ES6 fat arrow notation?

Re: TurboSphere

Reply #483 –
That would happen eventually anyway. To be honest, I'm not too keen on a lot of ES6. A lot of it sounds like things that people who don't really understand JavaScript think it needs. With the exception of a few small bits, like 'let' and string templates, I haven't really kept up too much on what engines can do what.

Maybe I will use SpiderMonkey. It's actually got a decent speed advantage right now (apparently TurboFan's teething problems are fairly significant), and looking at the docs, a lot of what I've needed to do by hand before is more ingrained in SpiderMonkey.

Plus, it can actually manage its memory, rather than just assuming the process will be terminated before too much garbage builds up like V8 does. And its got proper multi-threading support. And a real interpreter to fall back on, so it works on all platforms out of the box.

I remember I said a long time ago that I wanted to use V8 because I trusted it would stay significantly faster than SpiderMonkey. That kind of hasn't been the case in the last year or two.

Re: TurboSphere

Reply #484 –
I've got a new build of the engine (including the configmanager) using SpiderMonkey. No plugins yet (that's the hard part!), but it can run scripts at least. I guess years of using V8 taught me something about JS engines.

I noticed right away that SpiderMonkey has much better control of garbage collection (including a 'zealous' mode and a 'debug' mode), which is nice. For all I saw, V8 basically had no garbage collector at all.
I also noticed that SpiderMonkey is much more advanced with regard to multithreading and proper threaded use. I can easily imagine adding asynchronous callbacks with SpiderMonkey, whereas this was going to be a huge pain with V8.

SpiderMonkey actually makes me think it wouldn't be so hard to have multiple games run simultaneously from a single instance of TurboSphere. I can't think of a good reason to do this, though, except that it would be cool. And it would prove a certain cleanness of the engine's design.

Of course, coming up with a working engine build so quickly was vastly helped by SpiderMonkey actually being documented. "Read the Source" is not documentation. Not even close.

The flip side is that SpiderMonkey is a bit more verbose than V8 is. I'm OK with this so far, though. It's very consistent, whereas V8 has a rather irregular API.

I also took the opportunity to add multi-context tools to the configmanager, as well as cleaning out a lot of crap from it.

Re: TurboSphere

Reply #485 –
YOu sound skeptical and/or resistant in your diction, why is that? Is it performance or the fact you had been such a fan of V8 for so long? To me it seems like a slam-dunk win over V8. Is it developer fatigue (:P)?
If you use code to help you code you can use less code to code. Also, I have approximate knowledge of many things.

Sphere-sfml here
Sphere Studio editor here

Re: TurboSphere

Reply #486 –
It's actually really exciting to be working on this. I am surprised by how very much of my knowledge of V8 is helping here. Getting quick results with such huge changes is really satisfying!

It's a little disappointing to switch away from V8 after so long, so I guess I am constantly thinking of things like, "OK, but will SpiderMonkey let me down with using ArrayBuffers like V8 did?", or "Will SpiderMonkey's API change constantly now?"

That's also why I'm taking everything that seems better with a grain of salt. I am bracing myself to be let down, and to switch back to V8. If this proves disastrously difficult, then that's what I will do. At the very least, many of my long standing gripes with V8 seem to not be issues with SpiderMonkey.

So I guess I'm just speaking with the same kind of ambivalence I feel over the decision. It looks great on the surface, who knows what waits below.

Re: TurboSphere

Reply #487 –
https://github.com/FlyingJester/TurboSphere/tree/SM

The SpiderMonkey branch is now live on Github.

No plugins working yet, but full error reporting and some of the engine proper's functions work. Error are also reported with the correct line number now, which never worked right in V8!

Re: TurboSphere

Reply #488 –
I've updated the plugin tools to support SpiderMonkey. Still a few issues, but things are shaping up.

SpiderMonkey has a much nicer concept of a JS class. In V8, you just make a template and then a constructor which is no different in concept than any other native call. In SpiderMonkey, you actually register a JS class with the engine.

While the V8 way could work fairly well, too, in the last year or so V8 has been moving away from all that, though. Chrome apparently doesn't properly use even those templates, instead defining all properties to an object whenever one is created, and V8 seems to be moving that way, too. Very logical for a prototype based language -_-

Re: TurboSphere

Reply #489 –
I kind of get the feeling V8 was never meant to be embedded.  It's definitely pretty clear that it's written to cater to Chrome and everything else is secondary.  Spidermonkey on the other hand has always been designed for embedding.  There's official documentation, it's self-contained in a dynamic library, and you used to even be able to get the standalone binaries for linking (don't know if this is still the case).

Re: TurboSphere

Reply #490 –
The ScriptFS plugin works now. When it comes to defining JS classes, SM is miles ahead of V8.
Sapphire will be a bigger challenge, but it really shouldn't be that bad. A lot of what I've had to do for it so far is removing abstractions I made that are now redundant with SpiderMonkey's API.

@Lord English: I totally agree. V8 is very hostile to non-Chrome needs.

As evident how, on several occasions, I tried to provide small patches that added the missing ArrayBuffer functionality to V8 in different ways, and every time was given a different reason why it was bad. That was a long running issue I had, and so did many other embedders.

SpiderMonkey just has the functions I need for that. Because they are a useful thing for a JS engine to have, whether Firefox specifically needs them or not.

I can also now properly add an ArrayBufferToSurface function. This was quite literally not possible with V8.

Re: TurboSphere

Reply #491 –
Quote from: Flying Jester – on

I can also now properly add an ArrayBufferToSurface function. This was quite literally not possible with V8.


I don't get it, is it because this is an ECMA6 Array Buffer going to a Sphere Surface, and on the C++ side you didn't have access to the array buffer or it's contents?

If that's the case then SM is definitely a huge win since I'm pretty sure you're intending to use ArrayBuffers quite thoroughly throughout, like for vertex maps and such.
If you use code to help you code you can use less code to code. Also, I have approximate knowledge of many things.

Sphere-sfml here
Sphere Studio editor here

Re: TurboSphere

Reply #492 –
Quote from: Radnen – on

Quote from: Flying Jester – on

I can also now properly add an ArrayBufferToSurface function. This was quite literally not possible with V8.


I don't get it, is it because this is an ECMA6 Array Buffer going to a Sphere Surface, and on the C++ side you didn't have access to the array buffer or it's contents?


It's because ArrayBuffers are single-use in V8. In V8, once you get the address of underlying memory of an ArrayBuffer, that ArrayBuffer can not be GC'ed anymore. Even worse is that until recently you could only get the underlying address once. You've either got to leak it and let script keep using it, or free it with no way to tell script that it's not usable anymore! Both totally unacceptable--either I've got a memory leak, or I can cause a segfault from script? How is this a useful API?

With SM, I can just get the ArrayBuffer's underlying memory buffer address without making un-GC'able. I don't see why this isn't the case for V8. It works fine in SM, and I've even hacked the same functionality into V8 a couple times. It works fine, it's how the API should work, but the V8 authors have some deep-seated fear of this.

This way it is simple to read a portion of a file representing a bitmap (like in a Sphere tileset, spriteset, or font), and totally from script, pass the resulting ArrayBuffer directly to an image. Fairly close to the efficiency of native, but still leaving it possible to modify the bitmap from script.

You can still get the contents, but since you can't just do a memcpy/std::copy from the native side, you have to actually iterate the ArrayBuffer using a JS array or a TypedArray, even in native. I tried to do that, since it's the 'correct' way with V8, but it is just horrifically slow. It's not an option for anything but a trivial example. That's why I am so disappointed with V8 ArrayBuffers, a lot of the advantages that having native memory available to script could have are made impossible by how V8 handles them. So it's technically possible, but it would actually be an ArrayToSurface function, applying the ArrayBuffer to a normal JS Array or iterating a TypedArray like a normal JS Array. This is unacceptably slow and complex. A map with 32 tiles, each 16x16, and a 4-layer playing field of 64x64 tiles, takes about 2 seconds to read this way on my Mac Book, it should be much, much less!

Quote from: Radnen – on

If that's the case then SM is definitely a huge win since I'm pretty sure you're intending to use ArrayBuffers quite thoroughly throughout, like for vertex maps and such.


ArrayBuffers are key to how I want to do I/O with TurboSphere, almost exactly how Sphere uses ByteArrays. It also allows for a lot of shortcuts when reading files (like above, read a bitmap from a file in one operation, turn it into a Surface with one operation, with only a single ArrayBuffer being passed around). So they aren't really much different from how ByteArrays are in Sphere 1.5, but they are obviously native to JS, and I intend to do much more I/O in script.

Re: TurboSphere

Reply #493 –
Quote from: Flying Jester – on

It's because ArrayBuffers are single-use in V8. In V8, once you get the address of underlying memory of an ArrayBuffer, that ArrayBuffer can not be GC'ed anymore. Even worse is that until recently you could only get the underlying address once. You've either got to leak it and let script keep using it, or free it with no way to tell script that it's not usable anymore! Both totally unacceptable--either I've got a memory leak, or I can cause a segfault from script? How is this a useful API?


Ouch, wonder how Chrome handles that? I almost don't wanna know...
If you use code to help you code you can use less code to code. Also, I have approximate knowledge of many things.

Sphere-sfml here
Sphere Studio editor here

Re: TurboSphere

Reply #494 –
I don't want to say that I know for certain, but I suspect the answer is related to how each instance of V8 in Chrome has its own process...

Sapphire now compiles. There are still a few bugs to work out, but right now file reading and (mostly) graphics are working again, this time with SpiderMonkey.
SpiderMonkey provides a way to add a callback when a value is set by script, without acting as a setter. It is just a callback that fires on the set, and can optionally modify the value that is being applied. V8 used to call this an Interceptor. Used to. They are either deprecated, or fully removed now.

This lets us easily have the Shape array of a Group or the Vertex array of a Shape be accessible as a proper JS Array from script, and any changes to that script be applied in native without extra calls from script or polling from native. This required extra caching of the JS array by native and full getters and setters with V8.

One thing that caught me off guard about SM is that it crashes if you try to set an exception with another exception pending, before returning to script. This isn't really an issue, but it's different than how I worked with V8. Normally you wouldn't want to do this anyway, but consider this example:

When we check that the arguments of a function are the correct type and that the argument array is the correct length, we set an error if they aren't. In functions with variable arguments, I previously just ignored the set exception for the function signatures that didn't match. In SM, this actually makes no difference if I also indicate that there is no pending exception from the native function (the flag for exceptions can be true or false regardless of whether there is a set exception). If I set the pending exception a second time (like if another signature check fails), then BAM! Crash.

Fortunately, I built a suitable capability into the plugin tools previously. The signature-checking function can have a flag set to indicate we should not set an exception if the sigs don't match. This was primarily intended for when there are optional arguments on the end, but it works just as well here.