I fixed the Image:rotateBlit() issue. It was user error, as I suspected: Allegro treats the X/Y for a rotated blit as the center of rotation, whereas Sphere apparently aligns it to the top-left. It should have been fixed a long time ago, but I forgot all about it once I started working on the map engine.
Moving on... I added pixel caching for Surface:getPixel(). It definitely helped, but it's still ungodly slow. It might help if you stopped creating a surface anew with each collision check. Allegro stores all bitmaps on the GPU, so each time you call Image:createSurface(), a new texture has to be uploaded. Not only that, but it's putting a lot (and I mean A LOT) of stress on the garbage collector: I added some logging for images, and... well, let the console output speak for itself:
Last 4 lines after entering test map:
[image 4731] Cache miss!
[image 4731] Creating new pixel cache
[image 4731] get_image_pixel() using pixel cache
[image 4731] Dropped pixel cache
Last 4 lines after ONE attack:
[image 5868] get_image_pixel() using pixel cache
[image 5885] get_image_pixel() using pixel cache
[image 5868] get_image_pixel() using pixel cache
[image 5885] get_image_pixel() using pixel cache
So during processing of a
single attack, the game created (and possibly destroyed, depending on how aggressive Duktape's GC decided to be) over 1,000 bitmaps. The thing died right away, so my guess is a few attacks got coalesced due to lag, but still: 1,000+ bitmaps! That's insane. That's literally insane. No amount of pixel caching I can do is going to help that. I honestly have no idea why--or HOW--it works so well in Sphere.
HOWEVER, full disclosure--I don't think this is the only issue. What exactly goes on under the hood when you swing the sword? I'm consistently able to drop the framerate to 30-40 fps on my i3 just by slashing at thin air, but I get no log entries that a pixel cache was created during this.
The last thing I have to fix is that windowstyle glitch. I'm not sure how to deal with that yet. I may end up switching to an OpenGL build of Allegro. The only thing that worries me is that OpenGL tends to be slow on Windows, certainly slower than D3D. But it would fix the windowstyle issue, so....
Edit: Yep, same issue with the slashes--simply slashing at thin air creates and destroys nearly 100 bitmaps. Sorry to say, this game is VERY poorly optimized.
If it were me, what I would do is call .createSurface() once for each relevant sprite frame and then cache those somewhere. This way you can just pull them from the cache and do .getPixel() on them with impunity. As long as the surface contents never change (which they shouldn't), you'll never get a pixel cache miss and Allegro won't have to upload hundreds of textures for every attack.