Skip to main content ↑ ↓
Topic: Sphere v2 API discussion (Read 891216 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

Re: Sphere v2 API discussion

Reply #120 –
Now that they're no longer buggy as of mS 4.8.2, I'd like to introduce everyone to the SoundStream object!  You can use this to feed raw audio data directly to a mixer.  For example, using the Aurora.js library I can play mp3 files in JS!

Code: (JS) [Select]
window = global;
RequireScript('@/lib/aurora.js');
RequireScript('@/lib/mp3.js');

let fs = new FileStream('@/music/chartreuseRewind.mp3', FileOp.Read);
let mp3Data = fs.read(fs.fileSize);
let asset = AV.Asset.fromBuffer(mp3Data);
let stream = null;
asset.get('format', format => {
	// note: Aurora.js always decodes to float32
	stream = new SoundStream(format.sampleRate, 32,  // 32bit = float32
	                         format.channelsPerFrame);
});
while (stream === null && Sphere.run());
stream.play(Mixer.Default);
asset.on('data', buffer => {
	stream.write(buffer);
	Sphere.sleep(0.005);  // run event pump
});
asset.start();

Re: Sphere v2 API discussion

Reply #121 –
This is amazing, and opens up the engine for so many possibilities when it comes to audio capability. :D

Edit: stuff like this could be pretty interesting for inclusion. https://github.com/mohayonao/timbre.js/
Edit 2: or this for MIDI + soundfont support, with plenty of modification I'm sure. https://github.com/skratchdot/timbre.soundfont.js/

Re: Sphere v2 API discussion

Reply #122 –
Could you explain briefly how the project matrices work? Any attempt to use them gives me a blank screen.

(I'm trying to find a way to implement a mode7 equivalent and thought projection may be the obvious answer)

Re: Sphere v2 API discussion

Reply #123 –
There's a bit of theory involved there that I feel is necessary to know in order to use them effectively, but I'll try to keep it short.

tl;dr: Starting with an identity matrix for the projection will not get you the expected results.

It's worth noting to start with that the default value of .transform for a surface is NOT an identity matrix!  The default projection is an orthographic transformation matching the surface's resolution, in other words:

Code: [Select]
// w, h = surface size in pixels
defaultProjection = new Transform()
    .project2D(0, 0, w, h, -1, +1);

As far as the GPU is concerned, the render target goes from (-1,-1) to (+1,+1), regardless of its actual resolution (this is called NDC, normalized device coordinates).  So if your projection is identity, that's the box you have to draw within to get anything seen.  The projection matrix tells the GPU how to get from world space ("where things are drawn") to NDC.

2D is easy: You pass to .project2D() the corners of a rectangle in worldspace that you want stretched over the screen.  To prevent distortion, the aspect ratio of the ortho-box should match that of the surface.

For 3D the viewable space is actually a pyramid (technically a frustum), since you get more in view the further away you are.  For that you pass to .project3D() a field-of-view angle, an aspect ratio, and the Z-coordinates of the near- and farplanes, both of which must be nonzero and positive.  Explaining any more here is out of scope of this post since you wanted me to keep it brief. ;)
 

Re: Sphere v2 API discussion

Reply #124 –
Thanks for the brief intro, I'm afraid I can't see what I'm missing though :(

Any chance you could make a demo that shows project3D working?

Anything I do with it just gives me a blank screen.

Re: Sphere v2 API discussion

Reply #125 –
It seems there's a bug where setting screen.transform doesn't get picked up right away.  You can work around it for now by drawing something to a surface right after setting the transformation, then the next time you draw to the screen the new matrix will get picked up.

Anyway, this might help illustrate how everything comes together, we'll rotate the entire screen and then project it into 3D:
Code: (javascript) [Select]
var wh = screen.width / 2;
var hh = screen.height / 2;
screen.transform = new Transform()
    .translate(-wh, -hh)
    .scale(1 / wh, 1 / wh)
    .rotate(0.2, 0.0, 1.0, 0.0)
    .translate(0, 0, -1.0)
    .project3D(90, wh / hh, 0.1, 2.0);

    // workaround for miniSphere bug
    new Shape(ShapeType.Points, new VertexList([{}])).draw(new Surface(1, 1));

  • .translate() by half the screen size.  We want to rotate the whole screen, so we have to move the origin to the center (instead of top-left).
  • .scale() fits the "world" within (-1,-1)-(1,1) so we don't have to put the farplane insanely far away from the camera.
  • .rotate() to rotate the entire "world" (the screen) by 0.2 radians (about 11 degrees).  This might reveal stuff that's drawn offscreen!
  • .translate() again, -1.0 along Z to ensure the rotated image fits within the frustum (we can't do the Z-translation in the first step because it would screw up the rotation).
  • .project3D(), to apply the projection.  This should always be the last transformation you apply!

Fun thing about the Z coordinate: while the near- and farplanes are specified as positive values, Z coordinates of things you draw should actually be negative!  0 is exactly at the camera (and will be clipped, because you can't have a nearplane at 0), and moves in the negative direction "away" from the screen.  That caught me off-guard for a bit when I first starting playing around with 3D.

Re: Sphere v2 API discussion

Reply #126 –
Quote from: Fat Cerberus – on
Fun thing about the Z coordinate: while the near- and farplanes are specified as positive values, Z coordinates of things you draw should actually be negative!  0 is exactly at the camera (and will be clipped, because you can't have a nearplane at 0), and moves in the negative direction "away" from the screen.  That caught me off-guard for a bit when I first starting playing around with 3D.

I suspect this is the cause of your black screen - since the nearplane can't be at zero (if you wonder why that is: mathematically it's the same thing as dividing by zero), anything drawn with z = 0 will be clipped out of existence when rendered (this is also why walls tend to disappear if you get too close to them in 3D games).

Re: Sphere v2 API discussion

Reply #127 –
To avoid any unpleasant surprises: Starting in mS 4.8.3, transf.rotate() takes an angle in degrees instead of radians.  It's not as mathematically "pure", but much easier for novices to understand.

Re: Sphere v2 API discussion

Reply #129 –
Quote from: Eggbert – on
What is the purpose of using radians over degrees?

Mathematicians like it?  I'm not sure really, they're awkward to use because 360 degrees is ~6.28 radians (i.e. 2*pi).  Apparently some trig identities work better with radians than they do with other angle measurements, but other than that I can't see any real advantage.  Especially for game development.

Re: Sphere v2 API discussion

Reply #130 –
Quote from: Eggbert – on
What is the purpose of using radians over degrees?
They make a lot of mathematics simpler - without them you end up having to multiple by pi/180 all over the place if doing any funky trigonometry or geometry.

They actually make a lot more sense than degrees it's just everyone is used to degrees.

As one simple example, draw two radii in a circle, measure the angle between them in radians and multiply by the length of one of them and you get the length of the arc between them (i.e. the distance around the circle's circumference between the two radii).

Re: Sphere v2 API discussion

Reply #131 –
Debating whether I should add a SphereFS prefix for temp files or not.  The prefix would be %/ and to maintain sandboxing purity, would be guaranteed by the engine to have nothing in it on startup.  Anything written into %/ would be deleted on shutdown.

The main thing that's stopping me is that I'm not sure how useful the ability to write temp files would be in practice, particularly for game development.  Any insights for me to help me decide?

Re: Sphere v2 API discussion

Reply #132 –
Quote from: Fat Cerberus – on
Debating whether I should add a SphereFS prefix for temp files or not.  The prefix would be %/ and to maintain sandboxing purity, would be guaranteed by the engine to have nothing in it on startup.  Anything written into %/ would be deleted on shutdown.

The main thing that's stopping me is that I'm not sure how useful the ability to write temp files would be in practice, particularly for game development.  Any insights for me to help me decide?
The time I've wanted to make temp files has been to enable storing images or sound files alongside other data as sphere could only load these as individual files and couldn't convert bytearrays into images/sound the only way to unpack a document containing such things was to read the binary data for each image/sound then write it out to an individual file and load it.

Re: Sphere v2 API discussion

Reply #133 –
A few questions on dataReader.js (I thought I'd use it for loading .rmp files rather than re-inventing the wheel again)

1. I think it's meant to have:
const from = require('from');
at the top, the latest version doesn't have this but it uses from in the _checkStructDescriptor function.

2. Did you remove the ability to have multi-tier objects/arrays? I was looking back around page 74 of the miniSphere development topic and you'd been using a somewhat complex object with branches and self references in it to read an .rmp file, I assumed that this was with an early version of dataReader.js but I can't see any facility for processing of self references or branches in the latest version, am I missing something? If the facility is gone/not returning that's fine I'll write my own just don't want to do it if it's already there somewhere.

Re: Sphere v2 API discussion

Reply #134 –
Thanks for the heads up, I'll fix the bug in 4.8.4.

As for the branches and backrefs, I removed them because they were difficult to document adequately.  So I just tried to make it work similarly to a C struct because that's easy to understand and explain.  If there's enough demand I could bring the feature back, though. ;)

I admit I was nice being able to load an entire .rmp file with one call...