Spherical forums

Sphere Development => Sphere General => Topic started by: Fat Cerberus on July 21, 2015, 06:36:14 pm

Title: Future of Sphere: Sphere Debugger
Post by: Fat Cerberus on July 21, 2015, 06:36:14 pm
Very recently, I learned that Duktape has full built-in support to act as a debug server.  This means it would be entirely feasible to implement a Sphere debugger right now.

Duktape's debug API and protocol are full-featured, with single-step, step-over, etc., the ability to evaluate arbitrary code in the current scope, view and modify variables, and more.  My question is, would anyone other than me be interested in this?
Title: Re: Future of Sphere: Sphere Debugger
Post by: Defiant on July 21, 2015, 08:44:22 pm
Sounds like a handy tool in my opinion.
Title: Re: Future of Sphere: Sphere Debugger
Post by: N E O on July 21, 2015, 10:51:52 pm
If I knew how to debug properly I would certainly be interested in this.

In all honesty, I feel Sphere should have had a "developer mode" like this with either emulator-style debug functionality or Firebug-like debugging a long time ago, but due to lack of resources and knowledge until now minisphere (and TurboSphere if Flying Jester officially decides to offer it) will likely be the first Sphere implementation.

Given that there's no previous use case for Duktape to base yours off of, feel free to choose whether it behaves more like WebKit's Remote Inspector or like some other JavaScript debugger like Venkman or something.
Title: Re: Future of Sphere: Sphere Debugger
Post by: Radnen on July 21, 2015, 11:17:27 pm
If it can be linked to Sphere's Studio's code editor, I for one am totally for it. You can create a standalone .exe with command line operations that the Sphere Studio uses. The added bonus? Your code would be hopefully cross platform and Windows users (for now) can get a GUI for it.
Title: Re: Future of Sphere: Sphere Debugger
Post by: Fat Cerberus on July 22, 2015, 01:23:25 am
I imagine it would be possible to integrate with Sphere Studio.  The way the Duktape debug API is set up is, it acts as a server with its own protocol and everything, and you just provide a communication channel between it and the debugger.  So it would be a simple matter of having the engine open a port on localhost and let the IDE connect to it to initiate debugging.  It would even be possible to debug remotely this way. :D
Title: Re: Future of Sphere: Sphere Debugger
Post by: Fat Cerberus on July 22, 2015, 12:57:55 pm
Well, I've done the requisite initial work in minisphere itself to support this.  It's essentially a thin wrapper over Duktape's own debug protocol, so it was ridiculously easy to get it set up.

How it works is, you pass the command line option --debug to the engine.  This causes the debugger to be attached just prior to calling game() (pausing JS execution), and the engine will then wait for a connection on TCP port 812.  Once a debug client connects, it can then control JS execution remotely through Duktape's (surprisingly robust!) debugging protocol.

Currently the Duktape interface doesn't provide source code, so there must be matching source available on the client side.  For an IDE like Sphere Studio, this is of course trivial, but it's something to keep in mind if anyone wants to design a command-line debugger.

What I think I'll do is, implement a proof-of-concept debugger in C# before trying to tackle the task of implementing it in Sphere Studio.  This will let me get familiarized with Duktape's debug commands and such too.
Title: Re: Future of Sphere: Sphere Debugger
Post by: Fat Cerberus on July 23, 2015, 02:12:31 pm
Wow, this is proving more tedious than I thought.  Not the debugger support itself, that's easy, it's just that Duktape's debugging API is set up using dumb streams for communication between the engine and the debugger, with no guarantee you have a full message available at any given time (it can do reads and writes piecemeal, write more than one message at a time, etc.).  Which means in order to interpret debug commands you have to constantly do trial parses on the data you have available to see if you can do anything with it and if not, you have to buffer the data until you can.

Complicating matters even more is that the protocol allows pipelining--more than one request can be outstanding at a time, and requests are unnumbered: responses are associated with requests by the order they are received in, necessitating maintaining state on top of the tedious message buffering.

Long story short, it's a nightmare.  Worse, there's no actual synchronous API I could use instead--all debugging has to be done through this ridiculous protocol.
Title: Re: Future of Sphere: Sphere Debugger
Post by: Radnen on July 23, 2015, 10:11:46 pm
Why would the Duktape team create something so unweildly? Wouldn't they have these same issues?
Title: Re: Future of Sphere: Sphere Debugger
Post by: Fat Cerberus on July 23, 2015, 10:16:07 pm
It seems to have been designed with TCP in mind, which has no concept of message boundaries, so Duktape itself doesn't make any such assumptions either.  That's the impression I get, anyway.
Title: Re: Future of Sphere: Sphere Debugger
Post by: Fat Cerberus on January 06, 2016, 01:46:23 am
Following up on this to mention that, for minisphere 3.0, I want to make a command-line debugger.  The Sphere Studio integration served as an experiment to see if the concept of a Sphere debugger could work.  It worked very well, but since I can't get Mono to run Sphere Studio, debugging is limited to Windows.  Adding a command-line debugger like gdb will allow debugging on any platform minisphere itself runs on.

Now... what should this new debugger be called?
Title: Re: Future of Sphere: Sphere Debugger
Post by: DaVince on January 06, 2016, 06:01:13 pm
I suggest the unceremonious "minisphere-debug". :P

Anyway, this sounds excellent. I'd use it.
Title: Re: Future of Sphere: Sphere Debugger
Post by: Fat Cerberus on January 07, 2016, 12:47:15 pm
Nah, I don't particularly feel like typing msphere-debug on the command line every time I want to debug a game.  I ended up going with "SSJ", continuing the theme of naming minisphere GDK tools after DBZ concepts that I started with Cell. :D
Title: Re: Future of Sphere: Sphere Debugger
Post by: FBnil on January 07, 2016, 04:36:17 pm
so... my hair won't go shiny yellow when I use SSJ?

By chance, do you know about the DBZ abridged series? (using the original footage, storyline gets scrambled up. super funny)
Title: Re: Future of Sphere: Sphere Debugger
Post by: Flying Jester on January 07, 2016, 04:59:06 pm
Oh I wouldn't go near him. He's got a nasty case of the explosions.
Title: Re: Future of Sphere: Sphere Debugger
Post by: Fat Cerberus on January 07, 2016, 07:09:56 pm

so... my hair won't go shiny yellow when I use SSJ?

By chance, do you know about the DBZ abridged series? (using the original footage, storyline gets scrambled up. super funny)


That was part of the joke behind the name actually: You get so pissed off at your code because it keeps crashing or whatever, that you go Super Saiyan.  Then you type ssj on the command line to debug it. :D

My nephew introduced me to DBZ Abridged a year or two ago, it's pretty damn funny.
Title: Re: Future of Sphere: Sphere Debugger
Post by: Fat Cerberus on January 17, 2016, 03:29:08 am
Good news: I have a functional proof of concept for SSJ.  All it can do right now is resume execution (which is automatically paused upon debugger attach) or detach (which terminates SSJ), but this proves the communication with Duktape is working properly.

The rest should be smooth sailing as it's just a matter of parsing the command line (which can be done with strtok) and implementing responses for all the Duktape notifications.  All the heavy lifting of encoding and decoding messages for communication with Duktape is done.
Title: Re: Future of Sphere: Sphere Debugger
Post by: FBnil on January 17, 2016, 10:17:57 am
 ;D
Title: Re: Future of Sphere: Sphere Debugger
Post by: Fat Cerberus on January 17, 2016, 02:48:40 pm
Proof of concept:

Code: [Select]

C:\src\spectacles-i>cell -l dist && start msphere --debug dist && ssj
Cell 2.99.812 Sphere Game Compiler x64
A scriptable build engine for Sphere games
(c) 2016 Fat Cerberus

Building 'C:/src/spectacles-i/' as dist.
Processing Cellscript.js rule 'sphere'... OK.
Compiling assets... Up-to-date.
Installing assets...
  scripts/main.js
  1 asset(s) installed
sphere -> 'C:/src/spectacles-i/dist/'
0 error(s), 0 warning(s)
SSJ 2.99.812 Sphere Game Debugger x64
A powerful JavaScript debugger for minisphere
(c) 2016 Fat Cerberus

Connecting to '127.0.0.1:1208'... OK.
Handshaking... OK.
  Connected to minisphere 2.99.812
  Duktape version is v1.4.0

[SSJ @ scripts/main.js:6] BREAK: in function 'eval'
ssj > r
Spectacles: Bruce's Story miniRT Console
Sphere API v2.0 (minisphere 2.99.812)

Initializing Specs Engine test harness
Loading testcases from 'brucesStory.js'
Added battle test 'rsbBalcony'
Added battle test 'rsbFinal'
Loading testcases from 'lastLucidan.js'
Added battle test 'temple'
[SSJ @ scripts/main.js:46] FATAL: Error: a pig ate everyone at 8:12
[SSJ @ scripts/main.js:46] BREAK: in function 'game'
ssj > lv
manifest = [value]
session = [value]
ssj > e manifest
{
   name: "Spectacles: Bruce's Story",
   author: "Fat Cerberus",
   summary: "Follow Scott Starcross in his quest to stop the Primus.",
   resolution: "320x240",
   script: "scripts/main.js",
   minimumPlatform: {
      recommend: "minisphere 2.1 or later",
      apiVersion: 2,
      extensions: [
         "sphere-legacy-api",
         "sphere-obj-constructors",
         "sphere-obj-props",
         "sphere-map-engine",
         "sphere-rng-object",
         "sphere-s2gm",
         "sphere-spherefs"
      ]
   },
   frameRate: 60,
   sceneRenderPriority: 99,
   logPath: "~usr/Spectacles Saga/Console Log.txt",
   consoleBuffer: 1000,
   consoleLines: 10,
   disableAnimation: false,
   disableBattles: false,
   disableSplash: true,
   disableTitleScreen: true,
   directory: "C:/src/spectacles-i/dist/"
}
ssj > q
Shutdown requested, sending detach request.
^C


I had to Ctrl+C out of it because there's currently a bug in minisphere where the engine doesn't send the Detach acknowledgement to the debugger before it shuts itself down, causing SSJ to block indefinitely waiting for a message.  But as you can see, it's otherwise working as intended. :)
Title: Re: Future of Sphere: Sphere Debugger
Post by: Fat Cerberus on March 05, 2016, 02:40:34 am
Welp, another Monster drink-fueled programming binge later, yet another awesome minisphere feature is done.  minisphere 3.0 final release will be March 28 (minisphere's first anniversary!), and then everyone will be able to go Super Saiyan to debug their JS code any time they want!  Okay, so technically it's a public beta so anyone can do that right now, but you know... :P
Title: Re: Future of Sphere: Sphere Debugger
Post by: FBnil on March 05, 2016, 03:16:55 pm
I'll drink to that! Can't wait...