Spherical forums

User-Made => Libraries => Topic started by: Radnen on December 19, 2013, 07:17:54 pm

Title: Link.js v0.4.2
Post by: Radnen on December 19, 2013, 07:17:54 pm
Link.js v0.4.2

GitHub Page: https://github.com/Radnen/link

Link.js is a very fast, lazy executed, query library. Inspired by Lazy.js and LINQ, it aims to make data manipulation easier to write and fast to execute.

Examples:
Code: (javascript) [Select]

// get n'th even number:
var numbers = [0, 1, 2, 3, 4, 5];
var n = Link(numbers).where(function(item) { return item % 2 == 0; }).get(3).toArray(); // returns [4]

// get enemy with lowest health:
var enemies = [{hp: 20}, {hp: 35}, {hp: 15}, {hp: 40}];
var n = Link(enemies).sort(function(a, b) { return a.hp - b.hp; });

// get a person on the map at a location (assume the x, y, and tile function are defined):
var n = Link(GetPersonList()).where(function(name) { return GetPersonTileX(name) == x && GetPersonTileY(name) == y; }).first();

// see if a particular person exists:
var n = Link(GetPersonList()).where(function(name) { return name == "player"; }).length() > 0;


With TypeScript things look a lot closer to LINQ (and more compact!):
Code: (javascript) [Select]

var array = [1, 2, 3, 4, 5, 6, 7, 8];
var evens = Link(array).where(n => n % 2 == 0)->each(n => Print(n));


Have fun writing your queries!

You may need Function.bind(), especially for legacy Sphere versions. For those using minisphere, you don't have to worry.

Code: (javascript) [Select]

Function.prototype.bind = function (self) {
    var args = Array.prototype.slice.call(arguments, 1), func = this;
    return function () {
        return func.apply(self, args.concat(Array.prototype.slice.call(arguments)));
    };
};
Title: Re: [code] Query.js for Sphere
Post by: Fat Cerberus on December 19, 2013, 10:30:33 pm
Neat, it's like LINQ for JavaScript. ;D

Although, doesn't JQuery already do all this?
Title: Re: [code] Query.js for Sphere
Post by: Radnen on December 19, 2013, 10:50:21 pm
I took inspiration from LINQ and jQuery, yes, but this isn't geared for the web, and so I think can be more applicable for Sphere games than jQuery.
Title: Re: [code] Query.js for Sphere
Post by: DaVince on December 20, 2013, 12:12:58 pm
Looks useful, good work! Another one of those things I'd use if I made a bigger game in Sphere. :)
Title: Re: [code] Query.js for Sphere
Post by: Radnen on December 20, 2013, 03:26:52 pm

Looks useful, good work! Another one of those things I'd use if I made a bigger game in Sphere. :)


I actually did use it in a small game, my LD28 game (a simpler version of this). I needed something to elegantly search through an array of people or enemies, and I kept using for loops to do it. Then I realized, what if Sphere had a LINQ-like thing or a jQuery-like cascading function call thing? That means I could then do searches in a single line of code, lifting the complexity entirely.
Title: Re: [code] Query.js for Sphere
Post by: Flying Jester on December 20, 2013, 06:00:11 pm
Makes you think.

I usually try to organize data so that it is easily searched using loops. But maybe that's why I have a hard time thinking of a simple way to use this--I'm inadvertently making things harder to do this way.
Title: Re: [code] Query.js for Sphere
Post by: Fat Cerberus on December 20, 2013, 11:20:46 pm

Makes you think.

I usually try to organize data so that it is easily searched using loops. But maybe that's why I have a hard time thinking of a simple way to use this--I'm inadvertently making things harder to do this way.


The problem with that approach is that you have to prioritize certain data over other for optimization purposes, and once your dataset gets large enough and your types of queries varied enough, you end up with all kinds of conflicts of interest when trying to do said optimization.  That is why LINQ was invented in the first place.  Better just to lay out the data in a way that makes sense and use a query library like this one when you need to sift through it.

I may have to find a use for this in Specs when I ever find the initiative to work on it again.  How is performance with this, Andrew?
Title: Re: [code] Query.js for Sphere
Post by: Radnen on December 21, 2013, 12:42:22 am

I may have to find a use for this in Specs when I ever find the initiative to work on it again.  How is performance with this, Andrew?


The performance is not as good as hand-rolling your own loops. Look at this:

Code: (javascript) [Select]

Query.addProcedure("getAt", function(obj, x, y) { return GetPersonTileX(obj) == x && GetPersonTileY(obj) == y; });
Query.addProcedure("get", function(obj, match) { return obj == match; });
var n = Query.from(GetPersonList()).get("player").getAt(x, y).exists();


versus the hand-rolled version:
Code: (javascript) [Select]

function IsPersonAt(name, x, y) {
    return GetTileX(name) == x && GetPersonTileY(name) == y;
}


So in fact that query is much more inconvenient than the function call. (So probably was a bad example). But notice if yu want different behaviors you have to rewrite the for loop each time, causing "code explosion" (writing similar bits of code to do slightly different queries because you can't just change the conditionals each pass).

But, for a decent example like this:

Code: (javascript) [Select]

// get all enemies whose hp is lower than a specified amount, and make them run away:
var enemies = [{hp: 20}, {hp: 35}, {hp: 15}, {hp: 40}];
Query.addProcedure("getLow", function(e, match) { return e.hp <= match; });
var n = Query.from(enemies).getLow(20).foreach(function(e) { e.runaway(); });


versus this:

Code: (javascript) [Select]

// get all enemies whose hp is lower than a specified amount, and make them run away:
var enemies = [{hp: 20}, {hp: 35}, {hp: 15}, {hp: 40}];
function Runaway(hp)
{
    for (var i = 0; i < enemies.length; ++i) {
        if (enemies[i].hp <= hp) enemies[i].runaway();
    }
}


We see the for loop version is still faster since we can pack in a lot during a single for loop. But it get's more complex, and harder to read. And due to code explosion, the query can be rewritten a hundred times, but new for loops need to be written to do other things. Query is not a speed tool, it's a neatness tool. The queries themselves are supposed to look better and read easier than the for loops that you would generate from them. Also, with less for loops, you lessen the overall sources of error.

Lets put it this way: it shines best during turn based battle systems, and for a battle system with less than a hundred opponents on either side, performance is not a concern. I'd gladly take a small performance hit if it does two things:

1. drastically reduce the overall LOC.
2. drastically increases readability.

Which both ups maintainability. ;)
Title: Re: [code] Query.js for Sphere
Post by: Fat Cerberus on December 23, 2013, 10:03:43 am
Yeah, I didn't expect it to be faster than a purpose-built for loop, that would be asking for the impossible.  :D  I just meant would the performance hit be crippling in any real world scenarios, which you seem to think no, so that's good to know. :). Not worth suffering from massive lag for a little convenience.
Title: Re: [code] Query.js for Sphere
Post by: alpha123 on December 28, 2013, 02:33:21 am
So this is kinda like Underscore, but optimized for Sphere? Neat. I should check if stock Underscore works with Sphere. Either way, I do like the LINQ-iness of this. I'm no fan of C# but LINQ is pretty cool.

A couple of things I noticed browsing through the source:

I highly recommend checking out Lazy.js (https://github.com/dtao/lazy.js). Not only does it offer a similar syntax, it is much more powerful and has performance that meets or exceeds plain for loops. It should run in Sphere, although I haven't tried it yet. The downside is it's slightly less trivial to extend to incorporate game-specific methods like getAt().

This is kinda useful, but IMO it's just another example of NIH syndrome in the Sphere community. It just doesn't do things as well as Underscore, Lo-Dash, or Lazy.js.
Title: Re: [code] Query.js for Sphere
Post by: Radnen on December 28, 2013, 03:28:46 am
Well part of anything NIH, is learning how to make one. ;) During the compo I didn't want to work with large, ugly codebases that may only half work (well if they were browser-centered, that is). I needed something I could understand and use, for fun. And I thought the community would like to see it.

1. Ok.
2. foreach is a C# keyword, I've grown to spell it that way.
3. Ok. (__define(G|S)etter__ are not ECMA standard though, but there are other ways.)
4. Efficiency is in coding time, not execution speed, I should make that clear. Ha!

Edit:
Well, I looked into Lazy.js and it looks awesome! Feature rich with a cool implementation method. :) But it is too much for small games to put a 5300 line file into a project (though I suppose one can just use the minified version). I think Lazy.js would work right out of the box though.
Title: Re: [code] Query.js for Sphere
Post by: alpha123 on December 29, 2013, 12:02:43 am
That is a very valid point. I certainly have "invented" something that already exists to learn. I'm just not sure that the community needs another library that effectively reimplementes established libraries, only with fewer maintainers and an inferior (although dramatically simpler) implementation. Query does have the advantage of being built to be extended with game-specific functions though.

Haha, I didn't remember Lazy.js is 5.3k lines. :P That may be just a teeny bit overkill for small games....
Lazy.js is definitely one of the coolest JS projects I've seen. I'm not really into lazy-evaluation type stuff (I have definitely backed off on hardcore functional programming), I just like how useful it is and how it manages to be quite efficient.

An interesting experiment would be to take the ideas from Lazy.js and implement just the barebones features that can then be extended with game-specific methods. That is basically just take Query's concept and adjust the implementation using ideas from Lazy.js.

Title: Re: [code] Query.js for Sphere
Post by: Radnen on December 29, 2013, 03:41:40 am
Alpha: worrying about efficiency and the fear of inferiority in your or other peoples code can make you quite simply unhirable. ;) Then again, if speed is critical; complain away!


An interesting experiment would be to take the ideas from Lazy.js and implement just the barebones features that can then be extended with game-specific methods. That is basically just take Query's concept and adjust the implementation using ideas from Lazy.js.


The thought definitely passed my mind, twice. I'll take a deeper look into Lazy. But the concept shouldn't be too hard: create iteration points when doing 'transforms' on the underlying data, and only iterate when iteration needs to happen.

Effectively you take functions like map, where, sort, etc. and compile them into one single pass. Then when you need to access the data after the end of a query, you run all those piled-on functions, and then retrieve the data. This effectively emulates those hand-written for loops that does everything in one shot. But it is slower than those hand-rolled functions due to abstraction overheads, but much faster than n loops for n operations. (Woah... This sounds similar to what an SQL optimizer aims to do with certain operations.)

It's one of those things that you don't see, and when you do it's like lights going off. Because Lazy.js is exactly what the description of Query is trying to do: Emulate what you'd do for hand-written for loops, but make it reusable and extensible.
Title: Re: [code] Query.js for Sphere
Post by: alpha123 on December 29, 2013, 06:03:46 pm
Warning: longish read ahead, probably disorganized thoughts. Readers should continue at their own risk.

I am not to be held responsible for any spontaneous sleeping, confusion, or other mental or physical harm that may come to the party privy to this information (henceforth known as the "reader") as a result of reading the forthcoming paragraphs nor shall I be liable for ego harm due to disagreements with the opinions contained therein, which do not necessarily reflect the views of the Sphere or web communities and are solely the views of the author, who does not intend damage to any parties involved but is not culpable for damages including but not limited to the aforementioned afflictions or any of the following such as damages to the reader's keyboard, damages to the reader's cat sitting on the reader's keyboard, potentially useless knowledge gained from the text, or eye strain caused from reading small text.


Alpha: worrying about efficiency and the fear of inferiority in your or other peoples code can make you quite simply unhirable. ;) Then again, if speed is critical; complain away!

Given that I'm currently hired quite happily I'm relatively sure that's not an issue. Disclaimer: I work as a web designer. Which is what I've always liked to do. I'm not a particularly good programmer (despite how much it may seem that I criticize your stuff, I'm quite confident you are a much better programmer) especially in anything except JavaScript and Ruby. :P I'm just kinda picky and know JS really well. Hence hopefully constructive criticism (really I suppose I should just skip critiquing bits of code and write a JS tutorial). Like it or not, the quality of JS in the Sphere community is far, far less than that in the web community. That said, yours is generally pretty good.

...Anyway, with that out of the way, I've worked on too many projects that have simply not cared at all about efficiency. Making something more efficient is way harder after it's written and especially if the original developer leaves. I prefer to put a lot of thought into it initially and come out with something both correct and efficient. It takes longer to get something working, but in my opinion (which should probably be taken with a tablespoon of salt) definitely pays off on in the long run.

I feel a little hypocritical right now since my favorite languages are Lua, Ruby, Io, and JavaScript.... xP (Note to readers: these are among the slower languages currently (except Lua which is pretty fast).) My excuse is speed is not necessarily the same as efficiency.

In summary, I just feel that people place too little value on efficient code. In Query, something as simple as Query(nums).filter(function (n) n % 2 == 0).map(function (n) n * 2).reduce(function (a, b) a + b) has to iterate through the array three times. Sure, that's quite fast on modern computers, but Query is a relatively low-level library intended to be used by games. Games tend to have rather strict performance requirements and may involve large arrays.

[/rant]

Quote

The thought definitely passed my mind, twice. I'll take a deeper look into Lazy. But the concept shouldn't be too hard: create iteration points when doing 'transforms' on the underlying data, and only iterate when iteration needs to happen.

The real problem that I can think of is making it easy to extend. Ideally the extension author shouldn't have to worry too much about the lazy evaluation, but I'm not sure that's possible to avoid.

Quote

It's one of those things that you don't see, and when you do it's like lights going off. Because Lazy.js is exactly what the description of Query is trying to do: Emulate what you'd do for hand-written for loops, but make it reusable and extensible.

You do "see" Lazy.js's work somewhat in that it requires the each() method to actually do anything. That's rather negligible though. IIRC Lazy.js also has some method to convert the internal sequence to an array, as opposed to just being able to use the result directly.
Title: Re: [code] Query.js for Sphere
Post by: Radnen on December 29, 2013, 07:10:02 pm
Alpha:
Well, part of the issue is for Query-like libraries you didn't even know efficiency until you saw Lazy.js, and how dare you compare my code to that behemoth! (I'm but a single man, in a single field.) Query.js is a heck of a step in the right direction than a lot of code others write for their Sphere games, and I intend for it to really help people write less code and do more. Perhaps raw speed it has not, but the fundamental design pattern is really inspiring. I should also mention nobody wants to hear that their code is inferior... I didn't even know Lazy.js existed, so excuse me for not making it The Best Way Possibleā„¢.

Also, as a slight nitpick I think your words sting a lot since you put adjectives like "incredible" before many words that makes them feel stronger than they are. Incredibly ugly, stupid etc. sounds harsher than a half-dozen other ways of writing something. It's like that quote from the Big Lebowski: "You're not wrong Walter. You're just an asshole".


My excuse is speed is not necessarily the same as efficiency.


I agree! But, you are right games do require raw speed too. But I think I got that covered with a compiled source like SSFML, or Jest's TurboSphere. Anyways, I am actively looking into Query.js to try and speed it up.

/No hard feeling's man.
Title: Re: [code] Query.js for Sphere
Post by: alpha123 on December 30, 2013, 12:42:02 am
I wrote a slightly nasty reply to your post (I'm not hugely partial to being called an asshole), but decided against posting it. I did not mean my post which you replied to to be inflammatory.

Instead I simply want to cover two things.



I'm getting irritated by your tone of voice writing. The Big Lebowski quote is simply not necessary here, nor is "apologizing" for not making something the best way possible. Sorry you didn't hear from me what you wanted. I'm not around to make you feel good about your code and have no qualms using adjectives that mean what I want.
Title: Re: [code] Query.js for Sphere
Post by: Radnen on December 30, 2013, 01:45:45 am
Quote
I actually think if (!(typeof func == "function")) is incredibly ugly, partly because it looks noisy and because that's exactly what the != operator is for. In case you hadn't noticed yet, I tend to be strongly opinionated. The "incredibly ugly" is not meant to be harsh on you, simply harsh on that bit of odd code. Try to detach from your code a bit.


I do admit, it was stylistically not the best choice to have done that. Just today I made the same remark about code style on this page: http://msdn.microsoft.com/en-us/library/cc221403(v=vs.95).aspx, it's pretty bad! :P

Quote

The second is your statement "I should also mention nobody wants to hear that their code is inferior." I think this is a very poor bit of reasoning and a useless statement in general. If you don't hear that your code is inferior in some way, you won't improve. One of the first things a graphic designer (or really anyone in a creative profession) learns is to kill one's work. The faster one can detach, look objectively, and realize that one's stuff is crap, the faster one gets better. That definitely applies to programming as well. Maybe it's painful to hear that one's creation has some warts, but those are simply spots that can be improved.


Well the inferior part can certainly be true, think on the Lewboski quote. I'm not calling you an asshole, remember it's Walter in the quote (though by extension I'm calling you the Walter). It's the idea behind the quote. See, my code is inferior, and I see it now - to Lazy.js - and feature incomplete - compared to the others - but as it stands it's not so bad. I just think it's how you say it. So perhaps it's not that people don't want to hear their code is inferior, they do, but not in such a spartan way. I don't recall Lazy.js calling Underscore inferior.

TBH: I dreaded the day you read this and remark on it. You might think that would encourage me to write better, but not until Lazy.js I had no clue how I could have sped it up. I legitimately thought it was a rather appreciable piece of code I wrote, even for your eyes. Remember, I'm the guy who shied away from using functional programming techniques in my own code (like foreach), and you railed me on that that one time. ;)

Yet... All things considered I still don't think what I wrote is bad or inferior in any way. And I can simply ignore you this time. :)
Title: Re: [code] Query.js for Sphere
Post by: N E O on December 30, 2013, 03:07:16 pm
...Okay.

Can we return to civility please before I lock this topic? I have a funny feeling it's going to get out of hand again if one or both sides start taking programming style critique too seriously, and I'm reasonably sure critique can be given objectively without devolving into ad hominem.
Title: Re: [code] Query.js for Sphere
Post by: Flying Jester on December 30, 2013, 10:01:14 pm

This is kinda useful, but IMO it's just another example of NIH syndrome in the Sphere community. It just doesn't do things as well as Underscore, Lo-Dash, or Lazy.js.


Don't take this the wrong way. This is, what I feel, the bright side of how Sphere has historically been built, and how this community has operated:

I'd argue that a certain amount of NIH is one of the things that makes this community great. On the one hand, we don't seem to fear digging in and building things from scratch, for whatever reason.
On the other hand, the whole Sphere-scene has always had a lot of what I call 'American' programming. Almost everything is built right here, for Sphere (consider that we have all our own resource file formats, Audiere and Corona, and long ago SphereScript). It might not be the best, fastest, or most efficient implementation, but we built it ourselves. Because that's what you do--an older American creed--you build things yourself.

You can waste a lot of time making things from scratch that turn out to be of lesser quality, true. But on the other hand, you can sometimes make things better. Sometimes just because you can make more assumptions about usage, sometimes because no one else seriously tried to reinvent it.

And sometimes you reinvent things just because you wanted to  :)
Title: Re: [code] Query.js for Sphere
Post by: Radnen on December 31, 2013, 11:08:36 pm
It actually turns out you really can't use underscore, etc. in Sphere 1.5/1.6, the shallow memory threshold cuts off anything interesting. Which means yes, they may work, and especially for Lazy.js, you can't do big data in Sphere.

So... a lightweight query library like mine is actually a very good way to go in Sphere. :)

Lazy.js does not work in 1.5/1.6, it errors when writing to variables that are named keywords (due to earlier versions of ECMA). It does however work in SSFML.

Here's the interesting thing:
Code: (javascript) [Select]

var t2 = GetTime();
for (var i = 0; i < 25000; ++i) {
var d = "";
var a = Query.from([1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20]).map(timesten).where(even).foreach(function(i) { d += i + ", " });
}
t2 = GetTime() - t2;

var t3 = GetTime();
for (var i = 0; i < 25000; ++i) {
var c = "";
var a = Lazy([1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20]).map(timesten).filter(even).forEach(function(i) { c += i + ", " });
}
t3 = GetTime() - t3;

Abort("Query: " + t2 + "\nLazy: " + t3);


The output:
Query: 1190ms
Lazy: 1656ms

Lazy.js is slower. But why!? Because it's not doing what it was designed for: big data (see below for more). Anyways this does go to show my code isn't that inferior, in fact throwing around the term inferior without seeing results is kind of a wrong thing to do. It works in older versions of Sphere and is faster in SSFML. That makes my initial premise true: it is indeed efficient. But! I still think there is a way to make it go faster, and work. I'm working on an experimental version of Query.js that works like a compiler (creates a tree/sequence of the functional logic, and then performs it in one pass). I'm seeing some promising results on big data in SSFML, but it needs more work to truly be efficient. It may not be better than Lazy (that guy's a genius) but it should still be much faster.

Now, the good stuff! On this:
Code: (javascript) [Select]

var d = "";
var a = Query.from(genArray(200000)).map(timesten).where(even).foreach(function(i) { d += i + ", " });
var c = "";
var b = Lazy(genArray(200000)).map(timesten).filter(even).forEach(function(i) { c += i + ", " });


a took 500+ ms
b took 300+ ms

So on big data, Lazy wins, but in a real time action game you want to do a lot of short data fast. So... Query turns out to work faster for that. Lazy is a data thing, Query is a game thing. Now, yes, there will come a time where you want to use Query for data - item lists, npc lists, etc. - So therefore, I'm working on an implementation that will work for larger data in Sphere.

Query2 is coming along nicely. Doing the above over 200000 elements, the times are:
Query: 531ms
Lazy: 363ms
Query 2: 447ms

So, I'm in a middle-ground so far. :)
Title: Re: [code] Query.js for Sphere
Post by: Radnen on January 01, 2014, 04:47:09 am
Query 2 is coming along nicely

One more benchmark to show how Query 2 handles stress (do a lot of work, but only request the first 3 items):
Code: (javascript) [Select]

var t1 = GetTime();
var b = Query.from(genArray(2000000)).where(even).map(timesten).where(even).take(3).foreach(add);
t1 = GetTime() - t1;

var t2 = GetTime();
var c = Lazy(genArray(2000000)).filter(even).map(timesten).filter(even).take(3).each(add);
t2 = GetTime() - t2;

var t3 = GetTime();
var a = Query2.from(genArray(2000000)).where(even).map(timesten).where(even).take(3).each(add);
t3 = GetTime() - t3;

Abort("Query: " + t1 + "\nLazy: " + t2 + "\nQuery 2: " + t3 + "\n");


Times:
Query: 3255ms
Lazy: 911
Query 2: 945

So, Query 2 here is what I'll consider on par with Lazy. The only reason it is a tad slower has to do with traversing the underlying data structure.

And here is the winner that kicks Lazy in the down-right ass:
Code: (javascript) [Select]

var t1 = GetTime();
var b = Query.from(genArray(2000000)).where(even).map(timesten).where(even).foreach(add);
t1 = GetTime() - t1;

var t2 = GetTime();
var c = Lazy(genArray(2000000)).filter(even).map(timesten).filter(even).each(add);
t2 = GetTime() - t2;

var t3 = GetTime();
var a = Query2.from(genArray(2000000)).where(even).map(timesten).where(even).each(add);
t3 = GetTime() - t3;

Abort("Query: " + t1 + "\nLazy: " + t2 + "\nQuery 2: " + t3 + "\n");


Times:
Query: 3887ms
Lazy: 2135ms
Query 2: 930ms

I... I really think I'm on to something magnificent here in how to handle large data with compiler techniques.

Thank you Alpha for motivating me. :)

Edit: The current Query 2 API (subject to immense changes):
Code: (text) [Select]

chainable:
take(n)   - take the first n results.
first(fn) - take first satisfying item, or if no function, the very first item.
map(fn)   - perform a map operation with fn.
where(fn) - perform a filter using fn as the predicate.
get(num)  - tries to get the indexed item.
uniq()    - filters out items so only a unique list remains.
last()    - grabs the last result.
-x- random()  - selects a random result. --- TODO

non-chainable:
each(fn)         - performs the query, running the results through a function.
toArray()        - perform the query, returning an array.
contains()       - performs the query, return true if it satisfies the predicate.
every(fn)        - perform the query, checking to see if all items satisfies the predicate.
reduce(fn, memo) - perform the query, reducing the results, starting at memo, or if not, the first item.


Hmm, I think I need a better sounding name than "Query 2". I'm thinking of "Dash".

Edit:
It's actually slower than Lazy right now. :/ Darn. Why: It's taking too much time to go through the chain. I'm amazed that Lazy.js is near equal to the hand-written for loop version (sometimes, faster).
Title: Re: [code] Query.js for Sphere
Post by: Radnen on January 01, 2014, 09:50:18 pm
Update, More Performance

So, it depends on the query structure if Query 2 is faster than Lazy... But they are now mostly the same speed.

Code: (javascript) [Select]

var arr = genArray(200000); // an array: [0, 1, 2, ..., 199999];

test.addTest("Query Performance", function() {
Query2.from(arr).map(add).where(even).map(add).each(noop);
}, 10);

test.addTest("Lazy Performance", function() {
Lazy(arr).map(add).where(even).map(add).each(noop);
}, 10);

Abort(test.run());


Times:
Query: 194ms
Lazy: 197ms

But if I order them differently, sometimes Lazy wins. So I need to work on optimizing Query 2, whose 'lazy evaluator' is not it's end-all. I can do compiler-like optimizations to it, by doing different things depeding on various states, such as going to a more efficient unique checker if the input is sorted or not, etc.

Code: (javascript) [Select]

var arr = genArray(200000);

test.addTest("Query Performance", function() {
Query2.from(arr).map(timesten).where(even).each(noop);
}, 10);

test.addTest("Lazy Performance", function() {
Lazy(arr).map(timesten).where(even).each(noop);
}, 10);

Abort(test.run());


Times:
Query: 144ms
Lazy: 184ms

Code: (javascript) [Select]

var arr = genArray(200000);

test.addTest("Query Performance", function() {
Query2.from(arr).where(even).map(add).where(even).each(noop);
}, 10);

test.addTest("Lazy Performance", function() {
Lazy(arr).where(even).map(add).where(even).each(noop);
}, 10);

Abort(test.run());


Times:
Query: 140ms
Lazy: 110ms

I just realized, Query 2 has a really fast mapper!

Code: (javascript) [Select]

var arr = genArray(200000);

test.addTest("Query1 Performance", function() {
Query.from(arr).map(add).map(add).map(add).where(even).map(add).foreach(noop);
}, 5);

test.addTest("Query2 Performance", function() {
Query2.from(arr).map(add).map(add).map(add).where(even).map(add).each(noop);
}, 1);

test.addTest("Lazy Performance", function() {
Lazy(arr).map(add).map(add).map(add).where(even).map(add).each(noop);
}, 1);

Abort(test.run());


Times:
Query 1: 687
Query 2: 283
Lazy: 497
Title: Re: [code] Query.js for Sphere
Post by: alpha123 on January 04, 2014, 02:55:21 am
Is that the best you've got?

Code: [Select]

function even(n) { return n % 2 == 0 }
function add1(n) { return n + 1 }
function div3(n) { return n / 3 }
function isInt(n) { return Math.floor(n) == n }
function noop() { }

var t1 = GetTime()
Query.from(range(500000)).where(even).map(add1).map(div3).where(isInt).foreach(noop)
t1 = GetTime() - t1

var t2 = GetTime()
Lazy(range(500000)).filter(even).map(add1).map(div3).filter(isInt).each(noop)
t2 = GetTime() - t2

var t3 = GetTime()
For(range(500000)).filter(even).map(add1).map(div3).filter(isInt).each(noop)
t3 = GetTime() - t3


Query: 2029ms
Lazy.js: 2225ms
For.js: 1488ms

Yes I'm doing black magic.

Incidentally, I'm surprised how fast Query manages to be. I guess I was a bit hard on it initially.


Thank you Alpha for motivating me. :)

You're welcome. Your progress has motivated me to try some stuff too. :)
As you can see, I've started on my own version as well. It's 1AM here and I need to get some sleep, but I'll post about it tomorrow.
Title: Re: [code] Query.js for Sphere
Post by: Radnen on January 04, 2014, 05:54:23 am
I started testing Query2 in an actual JS environment (Google Chrome) and you wouldn't believe the speeds:

On map -> where, Query2 is about 6 times faster than Lazy.
Query 2: 34ms
Lazy: 214ms

On where -> map, Query2 is about 3 times faster than Lazy.
Query 2: 46ms
Lazy: 123ms

HINT: Query 2 turns map-where and where-map into a single "instruction" during it's optimizer phase (also where-where and map-map). The downside to this is the optimizer can take a while to perform, so the above was used with cached values, ie:

Code: (javascript) [Select]

var arr = genArray(25000000);

var Q = Query2.from(arr).map(add).where(even);
var L = Lazy(arr).map(add).where(even);

test.addTest("Query", function() {
var n = Q.each(noop);
}, 10);

test.addTest("Lazy", function() {
var n = L.each(noop);
}, 10);


That effectively prevents the optimizer from kicking in each time.

BTW, on take() they perform at the same speed, which is critical: you don't want to do a million things before you realize you only needed 10 items.

Soon, I'll release this to the public and get fame and glory and what-not. :P But right now it's still very rough and mad-scientist-like at places.

edit: On the pattern you had matched:
where(even) -> map(add1) -> map(div3) -> where(isInt)

I get the times on a range of 25000000 ints:

Query 2: 93ms
Lazy: 631ms
Query 1: 1210ms

Query2 is 6.7 times faster than Lazy, and 13 times faster than Query 1. :)
Title: Re: [code] Query.js for Sphere
Post by: alpha123 on January 04, 2014, 12:08:14 pm
*jaw drops*

What new devilry is this?

I'm really impressed Radnen, and quite eager to see the source code of Query2.

In the latest Chrome with an array of 25000000 numbers, I get:
Query: 2137ms
Lazy.js: 826ms
For.js: 156ms

5.3x faster than Lazy.js and 13.7x faster than Query1. May the speed games begin. ;) Looks like I have some work to do. :P

For.js takes a completely different approach from Query2. It takes all the method calls and compiles a plain-JavaScript for loop from them. This, surprisingly, is actually not as much faster than Lazy.js as I thought, much less Query2. The filter -> map -> map -> filter test case I gave is actually contrived to be very advantageous to For.js. It is capable of inlining the even, add1, div3, and isInt functions right into the compiled loop. JavaScript function calls have a fairly big overhead (especially in Sphere), but I never knew just how significant it was until I tried this. Obviously it's not enough to be noticeable in normal usage, but when iterating over huge arrays I was able to shave off several hundred milliseconds.

Here's an example:
Code: [Select]

For(nums).filter(even).map(add1).map(div3).filter(isInt).each(noop)

gets compiled into
Code: [Select]

function (s, l, b, p, f, a) {
  for(;a=s[b],b<l;++b){if(!(a%2==0))continue;a=a+1;a=a/3;if(!(Math.floor(a)==a))continue;f(a,b)
}

Note the lack of any reference to even, isInt, etc. This only works for a pretty small class of functions, but they happen to be the ones usually used for map, filter, reduce, every, etc; i.e. one-statement functions that aren't closures.

Now the question remains: What on earth did you do to get 6.7x faster than Lazy.js!?

EDIT: I gave it some testing and the inlining saves around 1100ms, give or take 100. So it's definitely a big win, but now I need to find some other way to lose another ~60ms.
Title: Re: [code] Query.js for Sphere
Post by: N E O on January 04, 2014, 02:23:46 pm

gets compiled into
Code: [Select]

function (s, l, b, p, f, a) {
  for(;a=s[b],b<l;++b){if(!(a%2==0))continue;a=a+1;a=a/3;if(!(Math.floor(a)==a))continue;f(a,b)
}



I already see a couple optimizations there, unless there's something above Sphere's JS that makes (!(a%2==0)) faster than (a%2!==0) or (a&1>0).



For div3 I've personally never seen a significantly measurable difference between a=a/3 and a=a*0.333333333333333333333333333333333 (or however many digits a double can go until precision is lost) so that one may be in its most optimized form.

In JS, I subscribe to the FBNil school of optimization ;)
Title: Re: [code] Query.js for Sphere
Post by: Flying Jester on January 04, 2014, 02:35:12 pm


  • The aforementioned a%2 optimization
  • add1 becoming ++a instead of a=a+1
  • Replacing Math.floor(a)==a with (a|0)==a (or === if one wants to truly be pedantic)



Can't say for Sphere's JS, but in V8 ++a and a++ are identical performance in almost every regard.
I would assume that a>>0 is faster than Math.floor(a), if only because of asm.js.
Title: Re: [code] Query.js for Sphere
Post by: alpha123 on January 04, 2014, 03:00:32 pm
@NEO: The reason it doesn't perform those is because those would require actually parsing the inlined functions. Currently it simply tokenizes them and does some very basic parsing to verify that they're safe to inline. It then does a little more parsing to replace variable names, but never at any point generates a proper AST. I could consider that, especially since in the end I'll likely be creating an AST of some sort from the method calls to then compile to the for loop. I currently think full parsing of functions would add significant overhead for relatively small gain, but if anyone can think of some really useful optimizations that involve rewriting the input to higher-order functions I could implement a parser and compiler.

Your first optimization is actually implemented now, because I restructured a bit of the compilation so that it avoids the continue altogether by wrapping the rest of the code in a non-negated if statement. The example now compiles to
Code: [Select]

function (s, l, b, p, f, a) {
  for(;a=s[b],b<l;++b){if(a%2==0){a=a+1;a=a/3;if(Math.floor(a)==a){f(a,b)}}
}

mainly because it's shorter (invoking the Function constructor has high overhead, so the less I feed it the better) and simpler (one less operation).

I believe n|0 is the fastest Math.floor()-type thing, but none of the bitwise hacks are quite equivalent to Math.floor() with regard to negative numbers. So it couldn't do that type of optimization automatically, although isInt() could be written with n|0 and For.js would still happily inline it.

I have been paying more attention than usual (i.e. none) to FBNil's School of Micro-Optimizations for this, because it's meant to be blazing fast even in Sphere 1.5. Still, I think at this point I'm likely to get bigger gains by doing less work, not by doing work faster. Whatever Radnen is doing, I imagine it's aggressively delaying evaluation to avoid doing much work. For.js does not do as much of that currently. His architecture lends itself more to that sort of thing; For.js would need some restructuring and probably a proper optimizing compiler to achieve that level. At the moment it relies mainly on clever and bizarre things like decompiling and inlining functions. Which if I may say so is pretty darn sick, but has only very small gains from this point and is of limited utility anyway since it's only possible for a few functions.
Title: Re: [code] Query.js for Sphere
Post by: Radnen on January 04, 2014, 03:43:51 pm
Alpha123: I had the exact same idea, but then I realized how much trouble it would be to create a feature complete thing like Lazy. Also, it's kinda like cheating. :P

My trick is to back out of execution when you no longer need it. I use the function stack frame heavily. My Where()'s are not as fast as Lazy's, but my WhereMap()'s are faster than Lazy's, it's on longer queries I get faster. I think for that instance your code may be faster than mine. It has, after all far more optimization potential, but at the expense of a lot of creator-struggle.

Oh, and in Sphere Query2 at least works, but it is flawed: Sphere's JS has bad stack-frame performance, so much so it can be worse that Query at times. So a for-loop generator like yours is perhaps the only best option. But in a good environment like Chrome, it's starting to be clear just how much faster than Lazy my code is by witnessing the speed gains from their more powerful stack frame handling.
Title: Re: [code] Query.js for Sphere
Post by: alpha123 on January 04, 2014, 04:15:52 pm

Alpha123: I had the exact same idea, but then I realized how much trouble it would be to create a feature complete thing like Lazy. Also, it's kinda like cheating. :P

In order to reasonably achieve feature-completeness and extensibility, For.js actually uses two types of methods: the ones that are compiled into the loop, like map() and filter()--these are the ones that have to be fast. For other, less common methods it simply calls out to regular functions, although that is not completed yet I had a prototype that was capable of it so it's at least doable.
I thought it would be a relatively easy way to get good performance too, until I was barely beating (and often losing to) Lazy.js even in Sphere 1.6. xP Then I got out the bag of JS black magic. I'm still amazed by how much function inlining boosts performance, even in a modern environment like Chrome. With function inlining off it's consistently ~200ms slower than Lazy.js on the 25m element array. Whatever Lazy.js does, it's very, very good. Since function inlining is fundamentally rather limited I need to come up with some more general optimizations to beat it, let alone Query2. Cheating is hard. :(

Quote

My trick is to back out of execution when you no longer need it.

I'm wondering if I can pull something like this off with aggressive usage of break and continue. The data flow in Query2 makes this a lot easier for you, I think.

Quote

It has, after all far more optimization potential, but at the expense of a lot of creator-struggle.

I think yours has more potential for algorithmic optimizations which are usually more effective. On the other hand, my type of inlining is simply not possible in your system, and it's a big win for the common case.
Oddly enough For.js is only around ~550 lines right now, and I think at least 30% of that is documentation and comments explaining what the heck I'm doing. Granted, the API is very small, but the core parts are in place. The creator-struggle hasn't bitten me quite yet (although perhaps the fact that at least 30% of the code is comments means it actually has...).

Quote

Oh, and in Sphere Query2 at least works, but it is flawed: Sphere's JS has bad stack-frame performance, so much so it can be worse that Query at times. So a for-loop generator like yours is perhaps the only best option. But in a good environment like Chrome, it's starting to be clear just how much faster than Lazy my code is by witnessing the speed gains from their more powerful stack frame handling.

Could you try trampolining the function calls to avoid creating extra stack frames? I'm not quite sure how your code works, but people have been able to sort of implement tail-call optimization that way. Emulating the stack in JS is rather expensive though, so you'd have to weight the benefits against some possibly large overhead.
Title: Re: [code] Query.js for Sphere
Post by: Radnen on January 04, 2014, 05:23:45 pm
Query 2 is only 429 lines, but expected to grow as more features are added.

I had thoughts of adding super-optimizations, like map-map-map and where-map-map-where, but I think that's a bit overkill, yet totally doable.

Query 2 does this:

A sequence like this: Map.Map.Where/Filter.Each();

Turns into this chain:

Map2Node -> WhereNode -> EachNode -> null

Map2Node is an optimization. Where/Filter type nodes breaks the chain early if they fail. This is not recursive or I'll run out of stack space, so it pops the chain back (via return) and restarts it, iterating to the next value. Now, it returns the successful value if it made it to an 'end node' like Each or All, or undefined if it stops part-way (Filter, Uniq etc). Checking for Undefined can aid in performing queries that involve 'contains' and 'every'. The optimizations work by reducing unnecessary node traversals and function stack creations. The optimizations speedups hit the limit of actually doing the raw for loop, and so with a sufficient optimizer I think the complexity is equal to a hand written for loop (with the only downside of traversing nodes added to the mix).

To run it, all I do is this:
Code: (javascript) [Select]

var i = -1, l = 25000000;
while (i++ < l && !Env.stop) r.exec(a[i]);


So it has a very lightweight loop on the inside, and the only overhead is node switching. Things like break and continue are handled with node chains returning early and setting the environment stop element to true.

Edit:
I was able to get Lazy to work in Sphere 1.6, it turns out the number of things I had to change was small.

How's the performance of the two? Well... The same! Everything I threw at it, they performed the same. For longer queries I notice Query 2 was always faster, but for single queries, Lazy won. To get an idea:

On map, Lazy was 1.3 times faster.
On map-map, Query2 was 2.0 times faster.
On map-map-map, Query2 was 2.2 times faster.
On map-map-map-map, Query2 was 2.7 times faster.

So, longer queries, especially the optimized cases with map-map, Query2 started to really haul ass. But, this is like 'best results' stuff and totally not typical. For typical queries (like yours with isInt and div3) the two performed the same (with Query2 slightly faster).
Title: Re: [code] Query.js for Sphere
Post by: Fat Cerberus on January 05, 2014, 12:52:48 am
So when will Query 2 be ready for primetime? I'd like to use this functionality in Specs, but I'm thinking it'd be better to wait for the latest version.  Unless Q2 will be API-compatible with Q1, in which case I guess I could just throw 1 in there for now...
Title: Re: [code] Query.js for Sphere
Post by: Radnen on January 05, 2014, 01:14:01 am
Use Q1 for now - definitely a good start. Then when Q2 is ready (which I may rename to "Dash" when it's done), you can seamlessly upgrade. Methods will also have various names: foreach, each, forEach; where, filter; uniq, unique; length, count, size; etc. So that just increases it's drop-in compatibility. But the .from() part of it may change; if it does I'd just wrap a compatibility thing around it that's all.

I had designed Query with turn based battle systems in mind. ;)

That said, Query2 does have all the core functionality working. It's current API (all complete except reduce and every):
Code: (text) [Select]

chainable:
- take(n)    - take the first n results.
- first(|fn) - take first satisfying item, or if no function, the very first item.
- map(fn)    - perform a map operation with fn.
- where(fn)  - perform a filter using fn as the predicate.
- get(num)   - tries to get the indexed item.
- uniq()     - filters the results to only unique items.

non-chainable:
- each(fn)         - perform the query, running the results through a function.
- toArray()        - perform the query, returning an array.
- contains(o|p)    - perform the query, returning true if it satisfies the predicate or matches the object.
- indexOf(o)       - perform the query, returning -1 if item not found, or the index.
- every(fn)        - perform the query, checking to see if all items satisfy the predicate.
- reduce(fn, memo) - perform the query, reducing the results, starting at memo, or if not, the first item.
- length()         - perform the query, returning the overall length.
- count(p)         - perform the query, returning the overall number of times the predicate was satisfied.


Reduce and every are not hard to implement, so they'll be done soon.

Next I have planned:
Zip,
Slice,
Reverse, <-- gonna be HARD in my very linear system ;)
Sort, <-- gonna be HARD in my very linear system ;)
Concat,
Union,
Intersection,
Difference,
First,
Random.

And that's all.
Title: Re: [code] Query.js for Sphere
Post by: Fat Cerberus on January 05, 2014, 01:36:31 am
What does zip do?  Don't think I've ever heard that term in the context of query systems...

And for what it's worth, I'm opposed to the rename to Dash.  It sounds like a codename.  Lazy and For.js make some sense; Dash is not self-explanatory at all, all it communicates is that it's fast, information which is irrelevant on its own. It'd be like calling TurboSphere simply "Turbo".
Title: Re: [code] Query.js for Sphere
Post by: Radnen on January 05, 2014, 01:56:29 am
Underscore, lo-dash, Dash. I think it's just another one of the family... I still haven't finalized the name though, I'm still calling it Query 2 so don't fret too much! :P BTW, map and reduce are done.

I just realized I can implement toArray with reduce. (But it's slower, so I won't replace toArray's functionality with that... Lazy should take note!).

Code: (javascript) [Select]
Query2.from(array).reduce(function(a, b) { a.push(b); return a; }, []);


Zip tries to insert items from another array into this stream. This is what Lazy does:
Code: (javascript) [Select]
Lazy([1, 2]).zip([3, 4]) // sequence: [[1, 3], [2, 4]]


It applies to lazy generation since you also want to delay the results of zip until you've performed your transformations.
Title: Re: [code] Query.js for Sphere
Post by: Radnen on January 05, 2014, 09:11:50 pm
I was able to make Query 1 faster than lazy. Jesus, Lazy must not be all that it's cracked up to be, or I'm missing something here.

My trick:

Again optimize with wheremap and mapwhere. Query1 now has those, but instead of an optimizer figuring that out for you, you do the conscious decision of adding it in.

Code: (javascript) [Select]

var a = Query.from(arr).wheremap(even, add1).mapwhere(div3, isInt).each(noop);
var b = Query2.from(arr).where(even).map(add1).map(div3).where(isInt);
var c = Lazy(arr).where(even).map(add1).map(div3).where(isInt).each(noop);


Query 1: 212ms
Query 2: 37ms
Lazy: 249ms

Though I will say this: Query 1 is a destructive method, it will modify the array. This is good in some situations and for games, even ideal since you may want to remove certain combatants. So I think Query 1 should always be used for game loops while query 2/lazy used for large data (esp. database-y stuff).

Lord English: I'd suggest you use Query 1 for a turn based battle system. ;)

Also, I was able to add sort to it. It's not ideal... But it's passable. The sort method is not lazy, it's eager. So when it encounters the sort method it stops everything and runs the query right there and then, then continues with the sorted data. Lazy doesn't sort arrays at all, you'd first create an array and then sort it yourself (preferably using JS .sort method), which is what I do anyways. So speed-wise they are no different. I would have liked my own sort method that works while in lazy execution, but it's hard since indices move around, etc.
Title: Re: [code] Query.js for Sphere
Post by: Fat Cerberus on January 05, 2014, 11:11:37 pm
I don't know, destructive traversal makes me nervous in any case, I prefer to have control over what stays and goes in the array.  Mainly because I prefer to keep my queries separate from maintenance operations.  Sometimes it's unavoidable and you have to modify the structure while traversing it but I try to avoid that whenever possible, mainly to minimize the number of expensive deep copies I have to do.
Title: Re: [code] Query.js for Sphere
Post by: Radnen on January 06, 2014, 12:21:58 am
You are right Lord English. It's usually better to find what get's deleted before you delete. So, there is that advantage. In that case, Query 2 would then be the only solution. But from a raw speed perspective, finding things to delete and then deleting them is a tad slower than finding and deleting at the same time; however you at least get no adverse side effects.
Title: Re: [code] Query.js for Sphere
Post by: Radnen on January 07, 2014, 07:43:41 pm
Here you guys go: https://github.com/Radnen/link

Use it, abuse it, tell me of it's issues. :)
Title: Re: [code] Query.js for Sphere
Post by: Radnen on January 12, 2014, 03:40:33 am
Okay... weird issue, that makes Link impossible to benchmark vs. other libraries.

Lazy is faster than everything - consistently - and I don't know how it is doing it. My benchmarks were not wrong for Link: the scores are indeed faster, when you run the benchmark program once. The second and third attempts it gets slow and I have no damn clue how or why.... No damn clue as to why. It can get up to 3 times slower on a second benchmark run.

Ok... so, Benchmark.js will do 'n' samples to get a good reading. That means link will run those 'n' times, and it is fast and produces good results. Then the very second time Benchmark.js is ran on it, boom it explodes and my Link library is consistently slower all the time.

There is nothing wrong with Benchmark.js - I have tried my own profiling code (not as fancy) but the same thing happens, the second benchmark run it gets slow. Not the second time Link runs it's query. Lazy produces no change in speed - at all!

The array is not being modified between each benchmark run, in fact it is recreated, but that recreation is not factored into the overall time.

I have a different version of Link that uses all arrays, and it is slower than Link, but still the same thing happens. The second benchmark run, it gets slower and Lazy remains unchanged. Whatever it is I have zero knowledge as to why this is happening. But it's literally making benchmarking impossible.

Edit:
Ok, I found the issue and it's out of my control. :/

Code: (javascript) [Select]

// A:
var link = Link(array).map(add).filter(even).map(add);

link.each(noop);

// B:
var link = Link(array).map(add).filter(even).map(add);
    link = Link(array).map(add).filter(even).map(add);

link.each(noop);

// C:
var link = Link(array).map(add);
    link = Link(array).map(add).filter(even).map(add);

link.each(noop);



Which of the above is the fastest? Well, technically all of them since link ends up doing the same thing, right? But it turns out that A is twice as fast as B and C lies somewhere in the middle. WTF?

Edit 2:
Ah ha! In Firefox this is not happening and I noticed two things:

1. Firefox's JS is 3 or more times faster than Chrome. Jesus Christ I had no clue. 0_0
2. Lazy is getting slower with each repeated call, yet Link gets faster.
3. But as I try different tests then come back, the same issue that plagued it under chrome also plagues it under firefox... Could it be a GC thing?
Title: Re: [code] Query.js for Sphere
Post by: Flying Jester on January 12, 2014, 05:21:47 am
Bear in mind, I haven't looked too closely at the guts of link.


Okay... weird issue, that makes Link impossible to benchmark vs. other libraries.

Lazy is faster than everything - consistently - and I don't know how it is doing it.


Perhaps it was optimized for implicit asm.js compliance? What JS VM are you testing on?


My benchmarks were not wrong for Link: the scores are indeed faster, when you run the benchmark program once. The second and third attempts it gets slow and I have no damn clue how or why.... No damn clue as to why. It can get up to 3 times slower on a second benchmark run.
...
Whatever it is I have zero knowledge as to why this is happening. But it's literally making benchmarking impossible.


The following may just be silliness. It's based on my slightly shaky theoretical knowledge of the internals JavaScript VMs. And I expect the problem doesn't really need quite this deep of an explanation.

If you are testing on newer JS VMs, it's quite possible that it gives the high-optimizer hell with highly variable types being given as function parameters. The high-optimizer gets a lot of positive reinforcement for a function taking a certain type, then you through all kinds of types into the function, and because of all the previous reinforcement it keeps trying to reoptimize the function for each new type. Like branch misprediction on a CPU.

Otherwise, it might be angering the garbage collector by reusing the same space for types that have different underlying representations. That can be difficult to see at first with V8, since it takes the 'sinking ship' approach and just grows the stack for a very long time. But a more efficient (size-wise) memory model (perhaps like SpiderMonkey), and it would have to reorganize the JS stack a lot. Which is hellishly slow.
Title: Re: [code] Query.js for Sphere
Post by: Radnen on January 12, 2014, 05:39:11 am
That's... not such a bad idea. But then again, this is how all functional programming libraries operate, they commonly change up the function definitions as you do different things. What I find interesting is Lazy is stable through all of that while Link is not... it varies, a lot.
Title: Re: [code] Query.js for Sphere
Post by: Flying Jester on January 12, 2014, 08:36:42 am
If you want 100% performance, you probably shouldn't pass arguments types that integral underlying implementations and vary type with function call?

That's what I've deduced, anyway.

The alternatives I can think of would be really ugly. You'd pretty much be reimplementing the VM's internal type hierarchy. Put that way, it sounds like it's completely the VM's job to make it fast.


Ah ha! In Firefox this is not happening and I noticed two things:

1. Firefox's JS is 3 or more times faster than Chrome. Jesus Christ I had no clue. 0_0
2. Lazy is getting slower with each repeated call, yet Link gets faster.
3. But as I try different tests then come back, the same issue that plagued it under chrome also plagues it under firefox... Could it be a GC thing?


The main functional difference between SpiderMonkey and V8 is how they deal with memory. At least, it's the biggest difference I know of.

SpiderMonkey is designed to try and not call GC that much, but to free memory on the spot more often when it knows it can.

V8, however, takes the 'sinking ship' approach. It grows the stack as much as possible, using all available memory. This is because it really is faster to just not free memory. That is, until you need to. Then it's really slow. But in the context of a JS engine, you often won't get to that point.

The fact that it has such a huge difference between SM and V8 makes me believe it is either some asm.js-based optimization kicking in (V8 doesn't do that, no idea why not), or it is related to garbage collection.
Title: Re: [code] Query.js for Sphere
Post by: Fat Cerberus on January 12, 2014, 06:07:55 pm
So Link is the new name for Query 2, I take it.  Clever, naming it after LINQ.  Now it really is LINQ for JavaScript!

I'll have to give this a test-run.
Title: Re: [code] Query.js for Sphere
Post by: Radnen on January 13, 2014, 03:57:19 am
So I made some adjustments so it can use less memory, and now I'm faster.

But there is still an issue and I really think it has to do with GC'ing. My code when benchmarked runs faster than Lazy (about 2 times faster) for the first 11 trials. Then on trial 12 it becomes 5 times slower, and then on trial 14 it gets 2 times slower, on trial 15 it is back to normal speed, about 2 times faster. Lazy remains constant throughout. :/

See for yourself:
Code: (text) [Select]

Link map-filter-map-each: 0.6099999998696148ms index.html:37
Lazy map-filter-map-each: 1.0999999986961484ms index.html:37
Lo-Dash map-filter-map-each: 4.469999999273568ms index.html:37
Fastest is Link map-filter-map-each index.html:40

Link map-filter-map-each: 0.6400000001303852ms index.html:37
Lazy map-filter-map-each: 1.1199999996460974ms index.html:37
Lo-Dash map-filter-map-each: 4.359999999869615ms index.html:37
Fastest is Link map-filter-map-each index.html:40

... snip ...

Link map-filter-map-each: 0.6700000003911555ms index.html:37
Lazy map-filter-map-each: 1.1199999996460974ms index.html:37
Lo-Dash map-filter-map-each: 4.360000002197921ms index.html:37
Fastest is Link map-filter-map-each index.html:40

Link map-filter-map-each: 2.169999999459833ms index.html:37
Lazy map-filter-map-each: 1.1299999989569187ms index.html:37
Lo-Dash map-filter-map-each: 4.370000001508743ms index.html:37
Fastest is Lazy map-filter-map-each index.html:40

Link map-filter-map-each: 2.1600000001490116ms index.html:37
Lazy map-filter-map-each: 1.1199999996460974ms index.html:37
Lo-Dash map-filter-map-each: 4.340000001247972ms index.html:37
Fastest is Lazy map-filter-map-each index.html:40

Link map-filter-map-each: 1.4199999999254942ms index.html:37
Lazy map-filter-map-each: 1.1300000012852252ms index.html:37
Lo-Dash map-filter-map-each: 4.339999998919666ms index.html:37
Fastest is Lazy map-filter-map-each index.html:40

Link map-filter-map-each: 0.6199999991804361ms index.html:37
Lazy map-filter-map-each: 1.1299999989569187ms index.html:37
Lo-Dash map-filter-map-each: 4.3299999996088445ms index.html:37
Fastest is Link map-filter-map-each


Link generates far more objects than Link does, but is somehow stable throughout. There is, I find, no apparent reason for Lazy's performance, when the lazy-evaluation methodology is the same for both link and lazy.

Edit: Okay, I think I know where the trouble is. See, a JS compiler needs a static function to optimize well.

So, it turns out doing this:
Code: (javascript) [Select]

function obj() {
    this.test = function() { /***/ }
}


Is worse than this:
Code: (javascript) [Select]

function obj() { }
obj.prototype.test = function() { /***/ }


New results are even faster (even 20+ trials later):
Code: (text) [Select]

Link map-filter-map-each: 0.5799999996088445ms index.html:37
Lazy map-filter-map-each: 1.1199999996460974ms index.html:37
Lo-Dash map-filter-map-each: 4.879999998956919ms


But now that means I lose private variable closures... Oh well.

Edit thrice:
I has released teh codez.

Benchmarks: http://radnen.tengudev.com/link-benchmark
Git: https://github.com/Radnen/link

Edit to the fourth:
I updated to v0.2.1, it is much faster now. :)
Title: Re: [code] Query.js for Sphere
Post by: Flying Jester on January 14, 2014, 02:42:02 am
Do you have an API listing for link? Or some simple explained examples?
Title: Re: [code] Query.js for Sphere
Post by: Radnen on January 14, 2014, 05:26:09 am
A good API is on the github page. Examples, well... you can try the ones from the front post. The semantics have changed.

Query.from(array).where(even).foreach(print);

Is now:

Link(array).where(even).each(print);

I could add a few examples to the github page, too. :) (I might rewrite the first post with Link details and kinda ditch Query altogether.
Title: Re: [code] Query.js for Sphere
Post by: Flying Jester on January 14, 2014, 05:30:24 am
Or just split off the topic, and leave the Query bit for posterity?  :)

Thanks. I'm definitely going to give this a try.
Title: Re: [code] Link.js v0.2.1
Post by: Radnen on January 14, 2014, 05:33:26 am
Nah, I'll just post Query as legacy code beneath it. Link has gotten good enough to be quite beneficial - I want to preserve the discussion on how I went from the slow piece of crap that is query.js to the magnificent link.js /gloat :P.

Edit: as of 0.2.2 the speeds are even faster:

Version 0.2.0:
Code: (text) [Select]

Link map-filter-map-each: 0.6099999998696148ms index.html:37
Lazy map-filter-map-each: 1.0999999986961484ms index.html:37


Version 0.2.2:
Code: (text) [Select]

Link map-filter-map-each: 0.18000000156462193ms (index):34
Lazy map-filter-map-each: 1.180000000167638ms (index):34


The stark speedup has a lot to do with the static function creation and function inlining things I've added since 0.2.0, oh, and the optimizations it makes. I'm loving Chrome now!
Title: Re: [code] Link.js v0.2.2
Post by: Flying Jester on January 15, 2014, 12:41:07 am
Here's a question:

What would the best way be, using Link, to remove all items from an array that have a specific value for certain properties?

EDIT:

It seems that uniq() likes to trim out objects that aren't actually duplicates (either by content or identity), too.
Title: Re: [code] Link.js v0.2.2
Post by: Radnen on January 15, 2014, 03:26:43 am

Here's a question:

What would the best way be, using Link, to remove all items from an array that have a specific value for certain properties?


Link works on a "copy" (not really, but it generates a copy as it works) of the array, so unfortunately it doesn't do any destructive changes. It's both a blessing and a curse depending on who you ask. So I'm a bit torn on that part of it right now since I myself am struggling with the concept of efficiently removing items. I might add in something like that, it may not be common to such libraries like Lazy, but you and me, seem to need it - since it's good for games.

Now, to delete things we can do this:
Code: (javascript) [Select]

// figures out what to delete, by populating an array of true/false values.
function deletable(a, b) {
    if (b needs deletion) a.push(true);
    else a.push(false);
    return a;
}

// runs the query:
var items = Link(array).reduce(deletable, []);

// performs the deletions:
for (var i = 0; i < items.length; ++i) { if (items[i]) { array.splice(i, 1); i--; } }


We need to figure out what to delete before we delete it, because deleting things as the query runs destroys it's stopping condition and can muddle up results. To do it I use reduce: reduce is used to create something. It takes two inputs from the original array and turns them into a single output (effectively reducing the size), however if you start it off with an array or object, we can do interesting things with values on an as-they-are-seen basis. The bad thing is we are effectively traversing the entire source array twice, which is no-good performance wise.

Also, notice there is no filter clause. It might be better to filter things out before calling reduce, but then you are working off a modified version of the array... a filtered version of it to be precise. So it'll tell you nothing as to where to snip an item out, sadly. (again hence the speed).

Ok... I convinced myself removing items is really terrible in these kinds of schemes... I can try to figure out deletion, but it get's tricky in the whole lazy-execution scheme. It might be doable, though.

Also, in my experience Array.splice is terribly, terribly slow. It turns out recreating a new array is better. Hmm...

Code: (javascript) [Select]

// populates an array with items that are okay to keep:
function deleted(a, b) {
    if (b does not need deletion) a.push(b);
    return a;
}

// runs the query:
array = Link(array).reduce(deleted, []);


The above might be faster if you are okay with replacing the original array with an entirely new one! (This now sounds more like a reduce-y type of thing to do). :)

In fact it get's even faster if we continue this line of thought:
Code: (javascript) [Select]

// runs the query:
array = Link(array).where(what-to-delete).toArray();


Now it'll be flying... Esp. if it's an array of objects. Since it doesn't need to recreate the objects, recreating an array is a low-cost alternative to hand-picking items to remove. It may in fact be the best solution, and what Link would do internally, anyways.

If you want to remove a single item... or only a few, we can do this:
Code: (javascript) [Select]

var bob = { name: "bob", dead: false };
var array = [bob, { name: "sue", dead: true }, { name: "ann", dead: false }, { name: "tom", dead: true }];

// runs the query:
array.splice(Link(array).indexOf(bob), 1); // straight-up removal.

// bob is now removed

// or use a new feature:
array.splice(Link(array).indexOf("dead", true), 1); // deletes an object with the first property of "dead" set to true.

// sue is now removed

// alternative, for deleting several items (also the safest solution since it doesn't assume the object is in the list):
var i;
while (i = Link(array).indexOf("dead", true) >= 0) { array.splice(i, 1); }

// sue and tom are now removed.


So there, I think I exhausted many possibilities! :)


It seems that uniq() likes to trim out objects that aren't actually duplicates (either by content or identity), too.


uniq is still a WIP, it filters out identical strings, and numbers only. I'll have to look into how other libraries do it. It might be cheating, but if the uniq doesn't work as other standard libraries do, it can mean bad things. I'm sure there is a good, general, algorithm somewhere. That said my current approach is not too bad, just a super-lite version of the real deal.

So, my current to-do list:
1. Implement skip()
2. Implement a destructive API - there may be no need...
3. Improve reduce
4. Improve uniq
Title: Re: [code] Link.js v0.2.2
Post by: Flying Jester on January 15, 2014, 05:06:51 am
I would be satisfied if, on objects, uniq only filtered out objects that had the same identity.

Alternatively, you could try something like:

Code: [Select]

var same = true;
for(var i in object1){
  for(var e in object2){
    if(object1[i]!==object2[e])
      same=false;
  }
}
Title: Re: [code] Link.js v0.2.2
Post by: Radnen on January 15, 2014, 05:40:34 am
Wait, are your objects made like so:

Code: (javascript) [Select]

var list = [p1, p1, p2, p1, p2, p3];
Link(list).uniq().toArray(); // := [p1, p2, p3]


Because then I can do reference checking which is 1000 times faster. True unique filtering is time consuming! I see all kinds of ideas online, and - even - yours will slow down a lot from going through their properties or recursing through their prototypes (etc).

Is:
Code: (javascript) [Select]

{ x: 0, y: 0 } == { x: 0, y: 0 } // yes
{ y: 0, x: 0 } == { x: 0, y: 0 } // yes?
{ x: 0, y: 0 } == { x: 0, y: 0 } (prototype different) // ex: new Point() vs. new Vertex()???
// What about different names for methods that do the same thing???


My code currently checks if the object is a property and, if so, filters it really fast. If not, it then puts it on a reference list and checks against that to see if the reference is being used. Now, if I employed your technique, it's going to take a lot of time to filter two unique items. A list of a hundred or a thousand may be the maximum for good operation. But 10000 items... and it's not so good anymore. It depends on how large of an object they are too!

The one other solution is to have a unique filtering callback:
Code: (javascript) [Select]

function my_filter(item, other) {
    for (var i in item) {
        var o = item[i];
        for (var e in other) {
            if (o != other[e]) return false;
        }
    }
    return true;
}

var array = Link(array).uniq(my_filter).toArray();


Then that way you can at least compare two items with your own method. It will run the callback against all other items that are objects to determine if it is indeed a unique element, for each element. This doesn't make it any faster, but at least you can fine-tune the check and compare against certain values. Then, then you can do something like this to speed it up a bit:

Code: (javascript) [Select]

var array = Link(array).filter(stuff).uniq(my_filter).toArray();


The filter above can filter out items with properties you know to be different among items to reduce your chances of uniq going over items that are largely different. It might help, it might not.

Edit:
If you know your items are of the same structure, this is faster:
Code: (javascript) [Select]

function my_filter(item, other) {
    for (var i in item) {
        if (item[i] != other[i]) return false;
    }
    return true;
}


And, so I think I'm going with the callback approach. It's really up to the user what they want to look for between two items that determines uniqueness. Do you need full recursive checks or surface level checks? Reference checks or just primitive checks? Prototypical checks or JSON-like stringify checks? There are no clear winners here how uniqueness is resolved. It's up to the data and what you intend to do with it.

Look out for v0.2.3 soonish. Ok, and it's up!
Title: Re: [code] Link.js v0.2.2
Post by: Flying Jester on January 15, 2014, 01:44:32 pm

Wait, are your objects made like so:

Code: (javascript) [Select]

var list = [p1, p1, p2, p1, p2, p3];
Link(list).uniq().toArray(); // := [p1, p2, p3]


Because then I can do reference checking which is 1000 times faster. True unique filtering is time consuming! I see all kinds of ideas online, and - even - yours will slow down a lot from going through their properties or recursing through their prototypes (etc).



Generally speaking, yes. That's what I initially thought uniq would check for.



And, so I think I'm going with the callback approach. It's really up to the user what they want to look for between two items that determines uniqueness. Do you need full recursive checks or surface level checks? Reference checks or just primitive checks? Prototypical checks or JSON-like stringify checks? There are no clear winners here how uniqueness is resolved. It's up to the data and what you intend to do with it.


A callback would be fine. In my case (and I would expect many others), There are certain properties I don't consider important for uniqueness. There are also a few cases where this would be a useful shorthand for when I would like to only check the uniqueness of a certain property or properties of the elements.
Title: Re: [code] Link.js v0.2.3
Post by: Radnen on January 15, 2014, 02:07:34 pm
Well, it does that now.

Uniq API:
Code: (javascript) [Select]

Link(array).uniq(); // does reference checks, or filters out primitives.
Link(array).uniq(test); // uses test to filter unique objects, or filters out primitives.


My next task is to put up a definitive API. I'll probably do that as a github wiki page.
Title: Re: [code] Link.js v0.2.3
Post by: Flying Jester on January 15, 2014, 02:42:55 pm
Here's something that would be cool. Could you add orthogonal foreach?

As in, a foreach where the function takes two arguments, each one an item from two lists, and it is called with every combination? Or would that be slower than just calling toArray on both Link objects and doing a nested loop?
Title: Re: [code] Link.js v0.2.3
Post by: Radnen on January 15, 2014, 03:10:15 pm
You might get away with using reduce, and go through all of the combinations:

Code: (javascript) [Select]

function stuff(a, b) {
    for (var i = 0; i < a.length; ++i) { b.prop += a[i].prop; }
    return a;
}

Link(array1).reduce(stuff, array2).each(action);


It wouldn't work on a list of primitives though since they are not passed by reference.
Title: Re: [code] Link.js v0.2.3
Post by: alpha123 on January 16, 2014, 01:13:31 am
You could do that with zip().

Code: (javascript) [Select]

Link(array1).zip(array2).each(function ([a, b]) {
  // do whatever
})


Would be slightly less elegant without SpiderMonkey's JS extensions, but you get the idea.

EDIT: Actually never mind, that does something different, I misread your post.
~Radnen: code tag fix
Title: Re: [code] Link.js v0.2.3
Post by: Radnen on January 18, 2014, 08:03:18 pm
Mother****:
Code: (text) [Select]

Link map-filter-map-each: 0.1900000000023283ms
Lazy map-filter-map-each: 1.1200000000098953ms
Lo-Dash map-filter-map-each: 4.499999999970896ms
actual for loop: 0.18000000003667083ms


I did a test, not expecting this in the slightest. Jesus Christ. No wonder Jest can speed up his calculations in his game; it's just as good as native for-loop speed, if only slightly slower.

The timer is not that accurate, and sometimes they are the same time. It's not quitting early or anything - it's actually doing the full range of calculations!
Title: Re: [code] Link.js v0.2.3
Post by: Flying Jester on January 18, 2014, 08:09:28 pm
It's a nice way to make run-time modification and redrawing of Majestic maps quite fast, and still very simple to read!

I would say that, if there are drawbacks of Link compared to Lazy or the like, Link is quite well suited to usage in Sphere.
Title: Re: [code] Link.js v0.2.3
Post by: Radnen on January 19, 2014, 01:59:32 am
tl;dr: Long rant gloat about how I "win", when really it's only for Chrome that I "win", but a good start nevertheless, and that I really do think Alpha has chance of blowing us all away.

Actually, I'm faster than a for loop sometimes. It technically shouldn't be possible, but I did a test and these two perform the same, and in fact the top one performs better on average (I think it knows 'noop' does nothing, and between Lazy and my library it may know that too):

Code: (javascript) [Select]

for (var i = 0; i < array.length; ++i) {
var result = add1(array[i]);
if (even(result)) noop(add1(result));
}


VS:
Code: (javascript) [Select]

for (var i = 0, l = array.length; i < l; ++i) {
var result = array[i] + 1;
if (result % 2 == 0) result += 1;
}


It seems counter-intuitive that the top one is faster since it must make calls, so this tells me that Chrome must be smart enough to inline functions anyways. which means Alpha's solution is happening for my code behind-the scenes anyways since my code most commonly resembles the first for loop.

So, therefore I might indeed have the absolute maximum in performance in my link library. I don't want to sound like a dick by saying that since I *know* Lazy tried hard and is far more flexible than my library. But if you want to see how one produces the fastest "querying library" this is it. This is the best... Otherwise you're just contending with raw for loop speed. If... if you can go faster than that, then you've succeeded in doing the impossible.

Closed the book, end of the road. Right here... For chrome. But, use Lazy if you want flexibility since this library has none of that. I can only see speed-critical, maximum performing code that consequently needs an upgrade to maintainability to ever use my library... And only there I can see a future for it. Sounds a lot like gaming, too. ;)

Of course, that is if you use Chrome. In FF and IE the experience is, fatser, but not as. And probably more noticeable in Sphere (might even be slower). It's one crutch is function calls. That's all... and Chome happens to have a good, fast solution for that, which my library leverages by incident.

So I digress, it might be possible to create a library good on all browsers, and in that case Alpha may have the right idea of generating the code for a for loop... Provided the construction itself is fast, and the generated code turns on any optimizations which I think doesn't. And is accepted in immature environments like IE. But it is really a promising lead in the best performance for such libraries.

Alpha: you've been quiet, I'm hoping you made some headway. And although I said yours sounds like cheating, well, technically I am. I'm interested in how your library performs outside of Chrome. :)
Title: Re: [code] Link.js v0.2.4
Post by: Fat Cerberus on January 27, 2014, 12:38:32 am
Can Link work with multi-dimensional arrays?  I have a few for loops that look like this one in my battle engine:

Code: (javascript) [Select]
for (var iList = 0; iList < unitLists.length; ++iList) {
    for (var i = 0; i < unitLists[iList].length; ++i) {
        var unit = unitLists[iList][i];
        unit.beginCycle();
    }
}


...which isn't too bad as-is, but it gets a little hairy when the cycle actually gets underway further down.  I'm wondering if there's any way to express such a loop with Link, or if I have to do some remodeling first.
Title: Re: [code] Link.js v0.2.4
Post by: Radnen on January 27, 2014, 01:45:40 am
Presently there is no such thing to search and filter 2D arrays, but you do bring up a good use case and I'll definitely start looking into it. That said since there is nothing to filter, I doubt Link would make the code any faster since filtering things is kind of it's strong suit.

Now onto the good stuff. Starting from Link 2.0 you can run multiple Link contexts, even inside each other, like so:
Code: (javascript) [Select]

Link(unitLists).each(function(item) { Link(item).invoke("beginCycle"); });


What's better is if you have any more 2D list processing you might get better results doing it the above mentioned way.
Title: Re: [code] Link.js v0.2.4
Post by: Flying Jester on January 27, 2014, 02:01:14 am
I'm going to have to give that try.
Title: Re: [code] Link.js v0.2.4
Post by: Radnen on January 27, 2014, 02:11:40 am
Ok, new feature Expand.

Code: (javascript) [Select]

var a = [[0, 1], [2, 3], [4, 5, 7], [6, 7]];
var s = Link(a).expand().filter(even).toArray(); // s = [0, 2, 4, 6]


I am still testing out the feature, but I think it's good to go. But some words of warning: .expand() treats the resultant list like a single dimension array. So, things like indexOf and toArray() will only give you the single array representation of their values in the end. It does this by design. I'll have to take a deeper look into how indexOf can return the index of a 2D element. But expect things like .each() and .invoke() to do exactly what you want them to do. :)
Title: Re: [code] Link.js v0.2.5b
Post by: Flying Jester on January 27, 2014, 02:22:22 am
Does it work with arrays of objects as well?
Title: Re: [code] Link.js v0.2.5b
Post by: Radnen on January 27, 2014, 02:25:40 am

Does it work with arrays of objects as well?


Not really. Why, is there a particular thing you want done with an array of objects that must use the object itself as an 'array-like' basket? I've been meaning to add more features for arrays of objects to it, but currently it's for arrays of arrays. It depends on what you want done, really.
Title: Re: [code] Link.js v0.2.5b
Post by: Flying Jester on January 27, 2014, 02:59:35 am
I actually meant 2D arrays of objects.

But now that you mention it, a nifty feature that I would certainly use if it was available would be to specify certain properties of objects as a target array. Like so:

Code: [Select]

function StrangeObject(x, y, l){
  this.x = x;
  this.y = y;
  this.l = l; //an array.
}

var a1 = [3, 4, 3];
var a2 = [59];
var a3 = [2, 4, 0, 1];

var arr = [new StrangeObject(0, 0, a1), new StrangeObject(0, 1, a2), new StrangeObject(1, 0, a3)];

var s = Link(arr).expandNamedProperties(function(a){return a.l;}).filter(even).toArray(); // s = [4, 2, 4, 0]



I have had uses for something like this in the past, usually when storing relationships between objects. For instance, a list of entities that were within a certain range of each other.
Title: Re: [code] Link.js v0.2.5b
Post by: Radnen on January 27, 2014, 03:18:03 am

I actually meant 2D arrays of objects.


Oh, yeah, Link can do 2D arrays of objects with .expand(), no problem. Now, this use case:


Code: [Select]

var arr = [new StrangeObject(0, 0, a1), new StrangeObject(0, 1, a2), new StrangeObject(1, 0, a3)];
var s = Link(arr).expandNamedProperties(function(a){return a.l;}).filter(even).toArray(); // s = [4, 2, 4, 0]


I have had uses for something like this in the past, usually when storing relationships between objects. For instance, a list of entities that were within a certain range of each other.


Is interesting. I could add an argument to expand, like so:
Code: (javascript) [Select]

var arr = [new StrangeObject(0, 0, a1), new StrangeObject(0, 1, a2), new StrangeObject(1, 0, a3)];
var s = Link(arr).expand("l").filter(even).toArray(); // s = [4, 2, 4, 0]


I think it won't be too bad. Without the prop name it'll expand a default array. I think this would be the most efficient.

Edit:

And so it is done, and now on the bleeding edge at GitHub. Keep coming with the ideas!!

Code: (javascript) [Select]

var a = [{a: [0, 1]}, {a: [2]}, {a: [3, 4, 5]}, {a: [6, 7]}];
var s = Link(a).expand("a").filter(even).toArray(); // s = [0, 2, 4, 6];


It occurred to me that you could emulate the above anyways with pluck and expand:
Code: (javascript) [Select]

var a = [{a: [0, 1]}, {a: [2]}, {a: [3, 4, 5]}, {a: [6, 7]}];
var s = Link(a).pluck("a").expand().filter(even).toArray(); // s = [0, 2, 4, 6];


I just added pluck, right now. :P
Title: Re: [code] Link.js v0.2.5b
Post by: Fat Cerberus on January 27, 2014, 10:51:27 am
What does pluck do, exactly?  Really thinking this thing needs some proper documentation.  The readme helps, but it's a bit lacking in its description of a lot of query operations.  For example, is there any difference between first() and take()?  They both look like they do the same thing.  Same for filter() and where().

And regarding my question above and the response: I'm aware Link won't make this particular use case any faster. However, the turn resolver in Specs isn't all that complicated to begin with so I'll gladly accept a bit of overhead in favor of improved readability and, more importantly to me, scalability. I don't have to filter anything now, but I can see myself having to do so later on (prioritizing certain battlers, etc.).  Besides, I'm a hands-on kind of guy so this will give me practice using Link in a real-world scenario.
Title: Re: [code] Link.js v0.2.5b
Post by: Radnen on January 27, 2014, 03:19:19 pm
Yeah, making a proper API page is on my todo list. Also, .expand() should be quite fast in fact since it clears up some overhead.

Pluck does this:
Code: (javascript) [Select]

var a = [{ prop: "hi" }, { prop: "cool" }, { prop: "stuff" }];
var s = Link(a).pluck("prop").toArray();
print(s); // s = ["hi", "cool", "stuff"];


Pluck will use the named property throughout the rest of the expression. Without pluck the resulting array would have just been the objects.
Title: Re: [code] Link.js v0.2.5b
Post by: Fat Cerberus on January 27, 2014, 05:17:30 pm
Just have to say, this thing is awesome.  My six-line for loop above turned into this:
Code: (javascript) [Select]
Link(unitLists).expand().invoke('beginCycle');


I'm loving it already!  In theory Expand can be chained as well for 3+ dimensional arrays, right?
Title: Re: [code] Link.js v0.2.5b
Post by: Radnen on January 27, 2014, 07:13:01 pm

Just have to say, this thing is awesome.  My six-line for loop above turned into this:
Code: (javascript) [Select]
Link(unitLists).expand().invoke('beginCycle');


I'm loving it already!  In theory Expand can be chained as well for 3+ dimensional arrays, right?


Umm, yes, of course! :)
Title: Re: [code] Link.js v0.2.5b
Post by: Fat Cerberus on January 27, 2014, 07:19:09 pm
Now the true test will be when I start doing some actual queries. :)
Title: Re: [code] Link.js v0.2.5b
Post by: Fat Cerberus on January 28, 2014, 12:07:03 am
Not a feature request, but I vote to rename/alias expand() to unroll().  "Unroll" makes it clearer what it's actually doing, especially when the results are treated as a one-dimensional array thereafter (i.e. flattening, or "unrolling" the original 2D array).  "Expand" suggests only that you're widening the scope of the search, which while technically true, doesn't paint the whole picture.  So yeah, Expand should be called Unroll. 8)

Oh, and while I'm here, I might as well ask.  How much of a performance hit am I taking by doing the following?
Code: (javascript) [Select]
var isUnitAlive = function(it) { return it.isAlive(); };
this.playerUnits = Link(this.playerUnits).where(isUnitAlive).toArray();
this.enemyUnits = Link(this.enemyUnits).where(isUnitAlive).toArray();


I do that at the end of a CTB cycle to remove dead battlers from the fight.  Originally, I was running a for loop through the unit lists and removing dead battlers in-place using splice().  Here, I'm recreating the whole array with a query.  I know that must be a lot more expensive, I'm just wondering by how much.
Title: Re: [code] Link.js v0.2.5b
Post by: Radnen on January 28, 2014, 12:51:14 am
AFAIK recreating an array is really quite fast. I mean, the GC hit is still 10x faster than calling splice each time. ;)

I'll add an alias to expand. I even added the alias expandInto when looking at arrays in objects. So, yeah I'll add unroll to it too.

Edit: and it is done.
Title: Re: [code] Link.js v0.2.5b
Post by: Fat Cerberus on January 28, 2014, 09:22:15 am
I guess that's true, since splice essentially rebuilds the array each time anyway.  There is one advantage to splice, however: it doesn't change the identity of the array.  Which I don't think matters in this particular case (I don't do any reference equality checks for these arrays), but important to keep in mind.
Title: Re: [code] Link.js v0.2.5b
Post by: Fat Cerberus on January 28, 2014, 11:34:49 am
Suggestion for new endpoint: none(), which does the opposite of every().  Right now you can get the same functionality by using every() with a negative predicate, but it would be great to be able to reuse the same predicate I use for where().  For example:

Code: [Select]
if (Link(this.playerUnits).none(isUnitAlive)) {
  // game over handling
}


isUnitAlive was a function I created to be used in earlier queries, so it'd be awesome to be able to re-use it for this test.  Currently I have to create a separate isUnitDead predicate and use every() with that.

Note that none() should probably return true if the array is empty.
Title: Re: [code] Link.js v0.2.5b
Post by: Radnen on January 28, 2014, 02:46:09 pm
Ok, it is done. I added none to the repository. No really the method, .none() :P

Since filter() had an opposite called reject(), I guess every() could have one too.
Title: Re: [code] Link.js v0.2.5b
Post by: Fat Cerberus on January 28, 2014, 03:29:45 pm
In my opinion, a library like this is all about improving readability and maintainability, preferably with minimal impact on performance.  Some amount of aliasing is expected.  reject(), for example, often makes the intent of a query more clear than a negative where().  Same for every() versus none().
Title: Re: [code] Link.js v0.2.5b
Post by: Radnen on January 28, 2014, 04:13:20 pm
Also I got an idea for keeping the identity of an array:

Code: (javascript) [Select]

var array = []; // filled with stuff, let's say.
var alive = Link(array).filter(theDead).toArray();
array.length = 0; // yes, this is a thing.
array.concat(alive);


I think it's faster to never use splice, clear out the original array, and push back in the entities that remain after an operation.
Title: Re: [code] Link.js v0.2.5b
Post by: Fat Cerberus on January 28, 2014, 04:50:55 pm
I would debate that point.  Your method essentially rebuilds the entire array twice. Given that, on average in the Specs engine, only one battler acts per cycle (in rare cases more than one, if multiple units happen to have an equal CV) and likewise only one unit generally dies in a single cycle, the splice operation amounts to rebuilding only half of the array on average (anything before the insertion point need not be touched), versus your two full rewrites above.  I fail to see how that is faster, unless some crazy optimization is happening behind the scenes I don't know about.

Granted all this is moot seeing as my battler arrays amount to 3 items each, max--beyond trivial to rebuild--but for huge arrays, if you're only deleting one or two items, splice is still going to be faster I would think.
Title: Re: [code] Link.js v0.2.5b
Post by: Radnen on January 28, 2014, 05:17:00 pm
No, yeah you are right. I completely forgot Link rebuilds the array with toArray and I wrote it! XD I've been using it so long I tend to forget things like that.

Now, what I could add is a remove point to it:
Code: (javascript) [Select]

Link(array).remove(theDead);


But I'm not sure if I like that or not. It certainly is an issue - keeping the same array identity, but changing it's contents.
Title: Re: [code] Link.js v0.2.5b
Post by: Fat Cerberus on January 29, 2014, 12:36:07 am
Just a quick question, can queries be built ahead of time and reused multiple times later, as in actual LINQ?  For example:

Code: (javascript) [Select]
var evens = Link(numbers).where(even);
assert(evens.none(odd));
if (!evens.contains(2)) alert("That's weird, there's no 2 here...");
if (!evens.contains(8)) alert("Did the number 8 get... ate?");
evens.each(function(num) {
  print(num);
});


That was... well, horrifically contrived, but you get the idea.

Edit: Now that I think about it, even better would be the ability to build a bare query ahead of time and then pass in an array only when you actually want to run the query on it.  That is to say, have the endpoints accept an array as argument, rather than the Link context.  This way you could run the same query on multiple sets of data.
Title: Re: [code] Link.js v0.2.5b
Post by: Radnen on January 29, 2014, 01:29:55 am
Yes, you can do exactly the above. Endpoints remove themselves as they return their data, so you can keep the same chain and use different endpoints on it.

Furthermore, you can change the target of a Link context by changing the .target member. It was initially designed to take an array as the first thing since other libraries are similar to that and I didn't want to break that heritage.

Code: (javascript) [Select]

var my_query = Link(array1).where(even);
my_query.each(noop);
my_query.toArray();

my_query.target = array2;
my_query.reduce(addUp);


I could add an array parameter to it but then that means rewriting a lot of code. I think setting the target is the best way.

But I must warn you, Link contexts do store some data, unique() is the most noticeable: after running unique() once it'll store what was unique, so subsequent uses of that Link chain will no longer provide you the same information twice. I'll have to look into a way of resetting certain nodes when queries are re-ran. Zip() also as an issue, it'll run only once and that's it, but I think they are the only two.

Edit:
I just now fixed zip and unique, and even skip to use a new internal 'reset' method for when re-running queries. Now query runs are 100% isolated.

What do you feel about a .retarget() method?
Code: (javascript) [Select]

var evenfilter = Link(array1).filter(even);

evenfilter.each(noop);

// later:
evenfilter.retarget(array2).toArray();


Then I have to ask myself should retarget change the original target or the target for only that run?

I would also like to see link-to-link interactions and have some super-lazy methods.
Title: Re: [code] Link.js v0.2.7
Post by: Radnen on February 01, 2014, 01:01:06 am
New feature:

Link.create(..., item);

Create n-dimensional arrays:
Code: (javascript) [Select]

var a = Link.create(10, 0);     // 1D array
var a = Link.create(5, 2, 0);   // 2D array
var a = Link.create(8, 8, 5 0); // 3D array
var a = Link.create(4, 3, function() { return new Item(); });


The first n-1 arguments represents the rank of the array, each value the range. The last argument is the 'fill'. If the fill is a primitive it'll be copied throughout. If the fill is a function it'll run the function each time. The function takes one argument: the index of the sub-array it is in.

Code: (javascript) [Select]

// array of: [[0, 1, 2],[0, 1, 2]];
Link.create(2, 3, function(n) { return n; });


This is good for creating a base array of which to do further work off of later. Makes multi-dimensional array creation not such a pain anymore. I think this multi-dimensional array handling is a good thing for Link over other libraries similar to it.
Title: Re: [code] Link.js v0.2.7
Post by: Fat Cerberus on February 01, 2014, 01:16:25 am
That's a neat feature.  The ability to pass a lambda was a smart move; I could see this being very useful for initializing lookup tables.

As for retargeting, I wouldn't lose sleep over it.  Now that I think about it even real LINQ has you specify a specific collection as part of the query, and for purposes of readability it's better to write queries on different arrays separately, the same way as if you were for-looping through them.

Just out of curiosity, why is this in the Resources forum instead of Libraries?
Title: Re: [code] Link.js v0.2.7
Post by: Radnen on February 01, 2014, 01:25:27 am

Just out of curiosity, why is this in the Resources forum instead of Libraries?


Because it used to be a simple code resource. :P

I can move it to libraries, and I think I should... and it's moved.


That's a neat feature.  The ability to pass a lambda was a smart move; I could see this being very useful for initializing lookup tables.


The lambda was essential. Sometimes you want to create a new object for each spot on the array. With a function you can. You can do other things like generate a Fibonacci sequence, too.

BTW, retarget is a completely optional feature. ;)
Title: Re: [code] Link.js v0.2.7
Post by: Fat Cerberus on February 01, 2014, 02:00:44 am

BTW, retarget is a completely optional feature. ;)


No no, I know that.  I just meant not to overthink the API for it, since it's not going to be needed at all 95% of the time.

Any reason why the lambda for Link.create only gets the immediate index as an argument and not a full set of coordinates?  Like I said, it sounds very useful for creating lookup tables on the fly, but you need to know all the indices to do that.
Title: Re: Link.js v0.2.7
Post by: Radnen on February 01, 2014, 02:06:51 am

Any reason why the lambda for Link.create only gets the immediate index as an argument and not a full set of coordinates?  Like I said, it sounds very useful for creating lookup tables on the fly, but you need to know all the indices to do that.


A simple error, I'll fix it. Thanks for pointing that out!

Edit: not so simple, I've been scratching my head for a while. Can anyone 10x smarter than me find how to pass n-arguments to the lamda expression, one for each index? I can do only 2D arrays, but never a generalized approach (so far).

Code: (javascript) [Select]

Link.create = function() {
var args = arguments,
stop = args.length - 1,
v    = args[stop],
isFn = (typeof v == "function");

function CreateArray(n, idx) {
if (n == stop) return (isFn) ? v(idx) : v;
var a = [], l = args[n], n = n + 1;
for (var i = 0; i < l; ++i) {
a[i] = CreateArray(n, i);
}
return a;
}

return CreateArray(0, 0);
}


It's recursive code, and I'm having trouble wrapping my head around a generalized method to find the individual indices of each array. I can do this for fixed arrays, like 2D and 3D, but somewhere in that recursive function is a way to get the index as it appears then just pass it along, just like I do for the first index.
Title: Re: Link.js v0.2.7
Post by: Flying Jester on February 01, 2014, 06:09:47 am
Try passing everything as arrays?
Title: Re: Link.js v0.2.7
Post by: N E O on February 01, 2014, 01:01:43 pm
Can anyone ... find how to pass n-arguments to the lamda expression, one for each index? I can do only 2D arrays, but never a generalized approach (so far).


You might want to read up on Function.apply (https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Function/apply) again for this one ;)
Title: Re: Link.js v0.2.7
Post by: Fat Cerberus on February 01, 2014, 01:09:32 pm
No, I know what he means.  Passing multiple arguments isn't the issue, the issue is keeping track of all the array indices across recursive calls.  I could probably figure it out, but I'm not on my laptop at the moment...
Title: Re: Link.js v0.2.7
Post by: Flying Jester on February 01, 2014, 09:44:47 pm
Passing the indicies in arrays would keep them as reference, rather than value.
Title: Re: Link.js v0.2.7
Post by: Fat Cerberus on February 01, 2014, 10:12:45 pm
Feature request: Ability to query against multiple arrays.  Thus the following:
Code: (javascript) [Select]
Link(arr1, arr2, arr3).each(function(item) {
    // ...
});

...would be equivalent to this:
Code: (javascript) [Select]
Link( [ arr1, arr2, arr3 ] ).unroll().each(function(item) {
    // ...
});
Title: Re: Link.js v0.2.7
Post by: Radnen on February 01, 2014, 11:52:45 pm
Ok, since arguments are slow to use to help speed it up I did a bit of trickery:

Code: (javascript) [Select]

var _splice = [].splice;

function Link(arr, test) {
if (!test)
return new Chain(arr);
else {
var a = _splice.call(arguments, 0, arguments.length);
return (new Chain(a)).unroll();
}
}


It should work naturally. I use splice on arguments since I've always known it as an array-like object, however, Link really only uses .length and the [] getter, that's it. But it could be an issue later on, so I've fixed it here. In JS 1.6 I could use an array generic to cast the list, but those methods aren't guaranteed to be implemented in any JS environment, well, besides Mozilla's, and I doubt Sphere will get to see them.

I put up v0.2.8b online, I'll mark it as officially 0.2.8 once I can figure out (or someone helps with) gathering the multiple array indices in the create method.
Title: Re: Link.js v0.2.8b
Post by: alpha123 on February 10, 2014, 01:31:59 pm
Radnen, would you mind if I used your proposed name Dash.js for my library (formerly known as For.js)? The fact that it compiles for loops behind the scenes is just an implementation detail now, and I don't feel like For.js fits it too well any more. I rather like the sound of Dash (it implies speed); if you don't mind, I think I'd rename it to that.

(Link.js is still, somehow, faster than it, BTW.)
Title: Re: Link.js v0.2.8b
Post by: Flying Jester on February 10, 2014, 01:41:45 pm
Huh, I thought it was called For.js because, when I read it, it reads as 'forges'.
Title: Re: Link.js v0.2.8b
Post by: Fat Cerberus on February 10, 2014, 01:57:38 pm
alpha123: There already exists a library called lo-dash (a pun on Underscore), naming it Dash might imply a nonexistent connection and confuse people.  I'm thinking that might be why Radnen chose not to use it.

Oh and Radnen: I think I might have a way to keep track of the array indices, but I have to test it out first, which I won't be able to do until later. Stay tuned! :)
Title: Re: Link.js v0.2.8b
Post by: Radnen on February 10, 2014, 02:05:31 pm

Radnen, would you mind if I used your proposed name Dash.js for my library (formerly known as For.js)? The fact that it compiles for loops behind the scenes is just an implementation detail now, and I don't feel like For.js fits it too well any more. I rather like the sound of Dash (it implies speed); if you don't mind, I think I'd rename it to that.

(Link.js is still, somehow, faster than it, BTW.)


Sure. Link is only very fast in Chrome, it is fast (faster than Lazy) in other browsers, but I don't think it's a universal solution. In Sphere it's not all that great (esp. on small arrays), so I've been using a different code called 'List.js' for arrays where I do one thing to it, and Link if I need to do more than one thing.

Of course, if those other browsers get faster, then Link will get faster. It also depends on how the code was written. In Firefox link was the fastest at one point, but then I changed the way I added methods, from:
Code: (javascript) [Select]

function foo() {
    this.bar = function() { };
}


To:
Code: (javascript) [Select]

function foo() { }
foo.prototype.bar = function() { };


And noticed a considerable speed up, with the bonus of not 'slowing down' on subsequent runs due to some weird compiler optimization thing in Chrome that Firefox never had. (Chrome was flagging the former functions as too much to optimize).


alpha123: There already exists a library called lo-dash (a pun on Underscore), naming it Dash might imply a nonexistent connection and confuse people.  I'm thinking that might be why Radnen chose not to use it.

I thought it was because you didn't really like it, and I've learned to trust your judgement on a few things. :P Plus, Link is not so bad, it is a lot like Linq.


Oh and Radnen: I think I might have a way to keep track of the array indices, but I have to test it out first, which I won't be able to do until later. Stay tuned! :)


That's cool! I just hope the solution is not too slow. If you have to manage a data structure that's anything more than linear to keep track of the  indices it may not be too worth it (for speed reasons it may just be better to use n for loops in that case). I'd like it to ideally create the arrays as fast as possible. ;)

So far I think my fastest solution just assumes at a finite 3D array and fills in indices only to those 3 slots. It might alienate larger arrays, but then the data becomes almost too specialized.
Title: Re: Link.js v0.2.8b
Post by: Fat Cerberus on February 10, 2014, 02:19:25 pm
Well yeah, that was true (and still is), I don't really like the name Dash.  It's basically just a pun on all the other similarly named libraries with no real meaning of its own other than "this is fast"--which is useful info, but not really the #1 priority with a library like this.  Otherwise you'd just roll your own for loops! :-). But yeah, I like the name Link.  It sounds like LINQ and is also meaningful: You're using it to "link" together related data that you'd otherwise have to sift through manually.

As for my solution, it's pretty simple, merely involving a 1D array (size equal to number of dimensions) and an integer variable.  And I don't think overhead is as big an issue here anyway since it's not like users will be calling Link.create() every frame or anything.  It's basically just to initialize a default data structure or lookup table, which for the most part should be a one-time thing.
Title: Re: Link.js v0.2.8b
Post by: alpha123 on February 10, 2014, 06:10:05 pm

Huh, I thought it was called For.js because, when I read it, it reads as 'forges'.

That's kind of clever, I hadn't thought of that. :P It was named because it was intended to allowed you to write for loops with a higher-level syntax, but really it just became a faster clone of Lazy.js with a completely different implementation and some cool features.


alpha123: There already exists a library called lo-dash (a pun on Underscore), naming it Dash might imply a nonexistent connection and confuse people.  I'm thinking that might be why Radnen chose not to use it.

I'm definitely aware of Lo-Dash, and the name Dash is actually slightly intended to imply that (mostly) nonexistent connection. ;P (They're pretty close API-wise, and the implementations are actually oddly similar in that they involve code generation and eval().) I thought Radnen didn't use it because Link implied that it's like LINQ, and you didn't like the sound of Dash.js. :P


Sure. Link is only very fast in Chrome, it is fast (faster than Lazy) in other browsers, but I don't think it's a universal solution. In Sphere it's not all that great (esp. on small arrays), so I've been using a different code called 'List.js' for arrays where I do one thing to it, and Link if I need to do more than one thing.

Huh, for me the difference between Link and other libraries is actually bigger in Firefox than in Chrome. I haven't tested my library in Sphere for a while and I'm not really sure how it will perform. On the one hand, since it generates code, it relies mainly on a fast eval() and that the generated code will be well optimized. Sphere doesn't really have either of those, but on the other hand, my library doesn't have a deep call stack, which Lazy and Link tend to create and Sphere generally doesn't handle too well.

Anyway, that is an odd thing that Chrome does with prototypes, and probably has to do with V8's "hidden classes" optimization thing. My library just assigns to prototype directly anyway, but that's good to know.


Well yeah, that was true (and still is), I don't really like the name Dash.  It's basically just a pun on all the other similarly named libraries with no real meaning of its own other than "this is fast"--which is useful info, but not really the #1 priority with a library like this.  Otherwise you'd just roll your own for loops! :-).

If you have any better ideas I'd be happy to hear them. To be honest 90% of the reason I like the name Dash is so that I can use this (http://static3.wikia.nocookie.net/__cb20130103074705/mlp/images/b/be/Rainbow_Dash_performing_Sonic_Rainboom_S01E16.png) as the mascot.... ;D

Incidentally, my library does roll its own for loops.... :P Hence the original name For.js, but I decided that it wasn't the greatest idea to name a library after its implementation (which is subject to change; I'm not sure generating for loops at runtime is the best choice in some circumstances).

Expect an alpha release of For/Dash/name-TBD soon. (Name suggestions are appreciated. :P) It's getting pretty close to having a full API; it's not nearly as comprehensive as Link's yet, but one of its main features is that it's fairly easy to extend.
Title: Re: Link.js v0.2.8b
Post by: Radnen on February 10, 2014, 07:04:12 pm

Expect an alpha release of For/Dash/name-TBD soon. (Name suggestions are appreciated. :P) It's getting pretty close to having a full API; it's not nearly as comprehensive as Link's yet, but one of its main features is that it's fairly easy to extend.


Yeah, my library is a pain to add things on to it. It's very "class heavy" - like how compilers are written. So adding a new feature means adding a lot of little stuff, most of which is boilerplate stuff that I can't seem to get rid of effectively. For example, some things have a .run method which kick-starts the query by staring off with the first modification made whether it is a map or a filter. But in order to do this for each type of operation the 'runner' as I call it needs to be written differently each time. The general runner just blindly goes to the next operation. It is still faster than Lazy, but for single operations is near Lazy's speed.

I guess you can still add stuff to it, but you end up doing a lot of the leg work:
Code: (javascript) [Select]

Link.prototype.add = function() {
    this.pushPoint(new AddPoint());
    return this;
}

function AddPoint(num) { this.num = num; }
AddPoint.prototype.exec = function(item) { this.next.exec(item + this.num); }

var added = Link([1, 2, 3]).add(1).toArray(); // 2, 3, 4


The above is how 80% of the library is implemented; which is why you can see how it is so fast. It runs at nearly the speed of traversing your standard every-day link list. That's the whole secret. I really want to expand to a binary tree, but such a format is not always easy to use when given an already linear array, but my hope would be to add binary searching to it - crazy stuff for a 0.3.0 release someday in the future.
Title: Re: Link.js v0.2.8b
Post by: Fat Cerberus on February 10, 2014, 08:06:33 pm
Haha, the name Link was even more clever than I thought, and I already thought it was pretty clever so that's awesome. :)

Also, brony alert! (Bronies are cool, I bear no ill will against them :) )
Title: Re: Link.js v0.2.8b
Post by: alpha123 on February 11, 2014, 11:07:49 am

Yeah, my library is a pain to add things on to it. It's very "class heavy" - like how compilers are written. So adding a new feature means adding a lot of little stuff, most of which is boilerplate stuff that I can't seem to get rid of effectively.

I know how that goes. My library pretty much is a compiler, actually (and a reasonably advanced one at that, capable of function inlining, loop unrolling, and specializing certain chains of functions). It does make adding new methods pretty tedious. To do your example, it would be

Code: (javascript) [Select]
For.prototype.add = function (num) {
    this.nodes.push(new AddNode(num))
    return this
}

function AddNode(num) { this.num = num }
AddNode.prototype.compile = function () {
    return 'a+=' + this.num + ';'
}

For([1, 2, 3]).add(1).toArray()  // [2, 3, 4]


which is surprisingly not all that different. It is, however, fairly tedious, so I developed an easier (but less performant) way to extend it by taking advantage of the fact that every array operation1 can be done as a map, filter, or reduce.2

Code: (javascript) [Select]
For.addFilter('even', function (element) { return element % 2 == 0 })
For.addMapper('add', function (element, index, amount) { return element + amount })
For.addEvaluator('sum', function (accumulator, element) { return accumulator + element })

For.range(10).even().add(5).sum()  // 45


All these methods do is create a function on For.prototype that does all the boilerplate stuff, and then calls map, filter, or reduce with a function that does more boilerplate and then calls the actual extension function.
There is another extension method, addFilterMap, which is a little more complicated (it generates a sequence of method calls instead of just a single one to map/filter/reduce) but allows both transforming and testing an element at the same time.

Quote

The above is how 80% of the library is implemented; which is why you can see how it is so fast. It runs at nearly the speed of traversing your standard every-day link list. That's the whole secret. I really want to expand to a binary tree, but such a format is not always easy to use when given an already linear array, but my hope would be to add binary searching to it - crazy stuff for a 0.3.0 release someday in the future.

I really like how that's implemented; it is simple, effective, and is very friendly to modern JS engines which will do a lot of optimization for you. It really is very fast too; I was only able to beat Link in Chrome by less than 10 milliseconds when I added loop unrolling. I still haven't beaten Link in Firefox, and this is in a test that is rather contrived to play off of my library's specific optimizations. :P
The binary tree thing would be quite interesting. I was thinking of extending my library to binary heaps and linked lists, but that would A) add a lot of complexity and B) I don't really know enough about data structures or algorithms to do that effectively.


Also, brony alert! (Bronies are cool, I bear no ill will against them :) )

Oh, alright, you found me. :D

1With some exceptions like drop and take. For/Dash/name-TBD's extension system doesn't quite support extending with methods like that yet, but the general groundwork is there.
2Technically I suppose map and filter can be implemented on top of reduce, but that would kill For's performance.
Title: Re: Link.js v0.2.8b
Post by: Fat Cerberus on February 11, 2014, 11:52:19 am
@Radnen:
Here ya go. :)

Code: (javascript) [Select]
	Link.create = function() {
var args = arguments,
stop = args.length - 1,
v    = args[stop],
isFn = (typeof v == "function"),
indices = [];

function CreateArray(n, i0) {
if (n == stop) return (isFn) ? v.apply(this, indices) : v;
var a = [], l = args[n], n = n + 1;
for (var i = 0; i < l; ++i) {
indices[n - 1] = i;
a[i] = CreateArray(n, i);
}
return a;
}

return CreateArray(0, 0);
}


As I said before, great for lookup tables:
Code: (javascript) [Select]
var timesTable = Link.create(11, 11, function(a, b) { return a * b; });  // JS really needs a lambda operator :'(
Abort(timesTable[5][5]);  // should be 25


Edit: Just sent a pull request on GitHub with the changes.
Title: Re: Link.js v0.2.8b
Post by: Radnen on February 11, 2014, 04:31:42 pm
Sweet, that's really nice. Thanks! :)


Code: (javascript) [Select]
var timesTable = Link.create(11, 11, function(a, b) { return a * b; });  // JS really needs a lambda operator :'(
Abort(timesTable[5][5]);  // should be 25



In Moz-land (JS 1.8+), you do have short-form lambda expressions in JS:
Code: (javascript) [Select]
var timesTable = Link.create(11, 11, function(a, b) a * b); // here.
Abort(timesTable[5][5]);  // should be 25


Edit:
I'm surprised how easy that looked. Just a single array incrementing a single value throughout the recurring chain. I like it. :)
Title: Re: Link.js v0.2.8b
Post by: Fat Cerberus on February 17, 2014, 10:26:24 pm

Edit:
I'm surprised how easy that looked. Just a single array incrementing a single value throughout the recurring chain. I like it. :)


Haha, yeah, I'm almost embarrassed how simple that turned out to be to implement.  It took some mental gymnastics on my part to come up with that solution only for it to turn out to be just two extra lines of code! :-[  Closures are awesome, aren't they?
Title: Re: Link.js v0.2.8b
Post by: Fat Cerberus on February 19, 2014, 02:09:46 pm
Hey, out of curiosity, what exactly do the .run() methods do?  I think you mentioned earlier in the thread how they were a cheat to enhance performance, but looking at the code for a few of them I don't really understand what exactly they're doing.  My best guess is that they're running parts of the query in advance and caching stuff to be reused later (for example, if the same chain is called twice), but as I said I'm not sure.  Care to clarify?
Title: Re: Link.js v0.2.8b
Post by: Radnen on February 19, 2014, 02:18:04 pm
It stops one less node traversal from happening. I don't know why, but it really ends up speeding things up considerably. On a filter-map-filter run, it would first go to filter, run it's node and move on. With a .run() method it'll instead start at map and run with a filtered result first thing. So in that case it's particularly fast.
Title: Re: Link.js v0.2.9
Post by: Radnen on March 07, 2014, 06:37:33 am
I released 0.2.9 it adds SQL-like features:



Now you can do this:
Code: (javascript) [Select]

// updating the coins of player named "Radnen":
Link(players).where('name', 'Radnen').update('coins', 15);

// a separated inventory table and player table getting player radnen's items:
var items = Link(players).where('name', 'Radnen').join(items, function(a, b) { a.name == b.name; }).select('name', 'items').toArray();

// retrieving only certain "columns" from a table:
var stats = Link(players).select('name', 'speed', 'strength', 'wisdom').toArray();


This is *not* an SQL implementation. It is rather SQL-like functionality built into Link. When you form queries you form them almost backwards than what you'd expect from actual SQL: you do your "selects" last and "wheres" first.

How is it *like* SQL you may ask? Well if you picture the rows as array entries and columns as object properties then you have basically the shim needed to do basic SQL-like transforms on the data. The 'players' table from above may look like this:
Code: (javascript) [Select]

var players = [
{name: "Radnen" , coins: 0  , strength: 1, wisdom: 10, speed: 5},
{name: "DaVince", coins: 20 , strength: 2, wisdom: 12, speed: 4},
{name: "Jest"   , coins: 100, strength: 1, wisdom: 11, speed: 6}];


And the items table could look like this:
Code: (javascript) [Select]

var items = [
{name: "Radnen" , items: ["helmet"]},
{name: "DaVince", items: ["bag", "jeans"]},
{name: "Jest"   , items: ["staff", "outfit"]}];


Join and select would work exactly as you would expect and update will modify the list.
Title: Re: Link.js v0.2.9
Post by: N E O on March 07, 2014, 09:26:24 am
Speaking of SQL-like queries, what's the likelihood of getting wildcards?
Title: Re: Link.js v0.2.9
Post by: Radnen on March 07, 2014, 12:58:00 pm
Such as select('*')? Yeah, don't use select and instead go straight to toArray(). in Link things are usually filtered out, so unless you limit what you take, select('*') is really the default behavior. For the LIKE clause, I could just employ a regex for all the other wildcards.
Title: Re: Link.js v0.2.9
Post by: DaVince on March 08, 2014, 02:16:49 pm
My god, Radnen, that is awesome. Makes things so much easier.
Title: Re: Link.js v0.2.9
Post by: Radnen on March 08, 2014, 02:42:23 pm

My god, Radnen, that is awesome. Makes things so much easier.


I particularly like the new .join method. Now I don't need super huge messy player data structures. I can strip out battle stats, inventory, "world stats", and other data to create more lightweight structures.

Now I can have player entities like this:
Code: (javascript) [Select]

// player-stats objects which are easily json-able:
function PlayerStats(params) {
    this.name = params.name || "unnamed";
    this.atk = params.atk || 0;
    this.def = params.def || 0;
    this.strength = params.strength || 0;
    this.wisdom = params.wisdom || 0;
    this.speed = params.speed || 0;
}

// player position objects:
function PlayerPosition(params) {
    this.name = params.name || "Unnamed";
    this.direction = "north";
    this.map = "";
    this.x = 0;
    this.y = 0;
}

// and a player-items object:
function PlayerItems(params) {
    this.name = params.name || "Unnamed";
    this.items = params.items || [];
    this.coins = params.coins || 0;
}


Then I can do join queries against several tables, if I were doing a multiple-entity game:
Code: (javascript) [Select]

// a player can only hold so many items based on their strength, we now have an easy way of getting this disjoint data:
Link(player_stats).join(player_items, function(a, b) { a.name == b.name; }).select('name', 'strength', 'items').toArray();


I'm sure there are plenty of other uses too. I mean, this really is best used with the fact that you have multiple party members.
Title: Re: Link.js v0.2.9
Post by: Radnen on March 15, 2014, 06:09:11 pm
Guys, here's a neat trick with Link. Did you know Link can unroll strings too? Accessing string characters is no different than accessing any standard array. So, here's a neat trick for finding a particular character in a set of lines. This is useful for playing a speech blip when certain characters appear on screen in a textbox.

Code: (javascript) [Select]

var ch = Link(lines).unroll().get(position);
if (ch != " " && ch != "\"" && ch != "\t") { PlayBlip(); }


It'll return the character at that position within the separated lines of text. Now, this is only useful if the lines were split from wordWrapString and you wanted to get what I call the global position of a character in the text. It's best used when 'position' is incremented upwards, starting at 0.

Basically I made this post to say, hey if you got lines of text Link works on that stuff too! ;)
Title: Re: Link.js v0.2.9
Post by: N E O on March 15, 2014, 06:43:30 pm
That is awesome. I may very well rewrite NTML using this...
Title: Re: Link.js v0.2.9
Post by: Radnen on March 15, 2014, 07:26:40 pm
Fir your NTML it might be useful to run markup through link like so:

Code: (javascript) [Select]

function CreateWord(word, array) {
    var type = 0;

    var ch0 = word[0];
    var ch1 = word[word.length - 1];

    if (ch0 == "*" && ch1 == "*") type = 1;
    if (ch0 == "_" && ch1 == "_") type = 2;
    if (ch0 == ":" && ch1 == ":") type = 3;

    if (type != 0) word = word.substr(1, word.length - 1);

    array.push({ word: word, type: type });
    return array;
}

var str = "This *is* a _marked up_ piece of :text:.";

var words = Link(str.split(" ")).reduce(CreateWord, []);


Now, once Link get's it's own split method we can go faster since it can split as it goes (currently the string is split at the beginning). So, this may not be ideal for large strings. Also, the above 'parser' is very simplified to get an idea of what one could do.

The operation with a Link.split method:
Code: (javascript) [Select]

Link(str).unroll().split(" ").reduce(CreateWord, []);
Title: Re: Link.js v0.2.9
Post by: N E O on March 15, 2014, 08:37:53 pm
Oh, I think you slightly misunderstood the domain of NTML; it's not conveniently aliased markup like Markdown or Wikitext, but instead more like HTML or BBCode.

I did restore NTML's wiki page (http://wiki.spheredev.org/NTML) a while back, so anyone who wants to can download the old, hopefully still working, version of the tech demo. NTML and the handler script are in dire need of updating to be cleaner and easier to extend in the future.
Title: Re: Link.js v0.2.9
Post by: Fat Cerberus on March 16, 2014, 01:19:01 am
Wait, why is the unroll() necessary?  A string is a 1D array of characters, it shouldn't even be possible to unroll it seeing as unrolling is for multidimensional arrays...

So yeah, I is comfuzed.
Title: Re: Link.js v0.2.9
Post by: Radnen on March 16, 2014, 06:25:22 am

Wait, why is the unroll() necessary?  A string is a 1D array of characters, it shouldn't even be possible to unroll it seeing as unrolling is for multidimensional arrays...

So yeah, I is comfuzed.


No, I'm not talking about a single array of characters, it's lines of text, so a 1D array of 1D strings, which can be unrolled. ;)


Oh, I think you slightly misunderstood the domain of NTML; it's not conveniently aliased markup like Markdown or Wikitext, but instead more like HTML or BBCode.

I did restore NTML's wiki page (http://wiki.spheredev.org/NTML) a while back, so anyone who wants to can download the old, hopefully still working, version of the tech demo. NTML and the handler script are in dire need of updating to be cleaner and easier to extend in the future.


Ah, my bad. My simple parser is a simple markdown-like syntax so I could write game dialogue easily, I hate writing escapable tags. I realized all that fancy text markup stuff is just a gimmick in the end and all I really need is basically one markup piece: emphasis. That and perhaps recoloring quoted text.

Saying: "Well he *said* he was going to get :the keys:."
Would be easier than: "Well he <i>said</i> he was going to <shaky>get</shaky> <color="yellow">the keys</color>"
Even if it was at reduced freedom, still 10x better to understand. (you might have differing tags, etc. but any *TML system would be similar.
Title: Re: Link.js v0.2.9
Post by: Fat Cerberus on March 16, 2014, 10:33:42 am
Ah, that makes more sense.  I think what threw me was in your original post, you said "Did you know you can unroll strings?"  I guess I just glossed right over the actual example since I was hung up on "wait a minute, you can't unroll a string, it's 1D".  Kind of put me in a trance I guess.  So yeah, my bad. :-[

Hey, out of curiosity, is Link prone to crashing Sphere?  Lately Spectacles has started causing random Sphere crashes, but the only major change I've made recently is the addition of Link, so...
Title: Re: Link.js v0.2.9
Post by: Radnen on March 16, 2014, 02:58:20 pm

Hey, out of curiosity, is Link prone to crashing Sphere?  Lately Spectacles has started causing random Sphere crashes, but the only major change I've made recently is the addition of Link, so...


What version? I know 1.6 crashes on things like windowstyles.
Title: Re: Link.js v0.2.9
Post by: Fat Cerberus on March 16, 2014, 03:46:20 pm
1.5.  I tried 1.6 a while back but it was too unstable so I was forced to go back to 1.5.  It's been rock-solid for me so far, but something I changed recently is causing engine crashes, so I'm trying to narrow it down.  It's times like this I wish Sphere had a debugger...

And I'm not using windowstyles so that's not it.
Title: Re: Link.js v0.2.9
Post by: Radnen on March 16, 2014, 04:08:15 pm
Hmm, it'd be hard to tell where exactly 1.5 is failing if it's Link, because that's purely a JS thing. Link should be stable even though it's still in development, the only thing I can foresee is that creating new link contexts each time could be the culprit:

Code: (javascript) [Select]

// this:
for (var i = 0; i < 1000; ++i) Link(array).filter().map().whatever().each();

// versus this:
var l = Link(array).filter().map().whatever();
for (var i = 0; i < 1000; ++i) l.each();


The top code may use more memory more frequently than the bottom code. What it *might* be is Sphere running out of JS memory in certain spikes, even if it's GC'ing it perfectly. It could be that at some moments it doesn't GC in time and breaches the max memory. What kind of crashes are you experiencing? Crash to desktop or is there an actual error message?

Edit:
I'm having the hardest time trying to implement a string .split method in this lazy format. The issue I'm having is splitting the last word. Take this example: "Hi how are you?".split(" "). It should split into the 4 words: "Hi" "how" "are" "you?", but once Link sees the '?' it stops execution at that point since it reached the end of the road. I only get back the first 3 of the 4 words. In a lazy context there is no easy way of detecting the end of the road. I could run the first n-1 items, then on the last item, item n, run a special condition saying you reached the end, and split there. But that's great until you start filtering things out. Then the true end position has changed, it might no longer be at the end of the originating array. I'll have to rethink my lazy strategy to account for this inconvenience.

I might even have to rewrite the entire library.
Title: Re: Link.js v0.2.9
Post by: Fat Cerberus on March 16, 2014, 09:49:15 pm
It's a hard crash to desktop.  No error message, Windows just suddenly throws up a crash dialog and closes it out.  I'm wondering if it is a memory breach as you said, since I do create a lot of Link contexts.  Nothing as extreme as your 1000-iteration loop, but I create a new Link context every time a battle unit requests a turn forecast, among other things.  Seeing as this often happens many times per ATB cycle (enemy AIs do it a ton and a new forecast is taken every time a different attack is selected in the menu)...

How much memory does Sphere allocate to JS anyway?  From what I've read around the forums I get the impression it's not a whole lot.
Title: Re: Link.js v0.2.9
Post by: Flying Jester on March 16, 2014, 10:03:25 pm
According to the dev log in the internal docs, scripts get 256k of memory.
Title: Re: Link.js v0.2.9
Post by: Fat Cerberus on March 16, 2014, 10:05:12 pm
Wow, that's tight.  I was expecting a couple MB at least, 256k is insane.
Title: Re: Link.js v0.2.9
Post by: Flying Jester on March 17, 2014, 01:04:37 am
Consider, though, that the size of every engine defined object is only a few tens of bytes at most on the script side. In fact, strings may be handled similarly.
Title: Re: Link.js v0.2.9
Post by: Radnen on March 17, 2014, 01:24:56 am
Also, Lord English, don't create a lot of on-the-fly anonymous functions in Sphere. They are notorious for consuming a lot of memory and sadly Vanilla Sphere's JS doesn't do any smart caching on them. This is also true for any nested functions (which should ideally be cached, but aren't).

Code: (javascript) [Select]

Link(array).where(function(i) { return i %2 == 0; }).each();

// versus:
var f = function(i) { i%2 == 0; };
Link(array).where(f).each(); // later on.


So, about my Link.split method, I don't think it's possible. Link only looks at the current state each iteration. I don't know how libraries like Lazy do it. They must have some cheap method of peaking ahead... Or they truly aren't lazy. Or .split is relegated to very specific use cases and break otherwise. I could do a lazy-evaluated split statement, but it can only be run as the first thing in a series of steps.

So, this is possible: Link(string).split(" ").filter().each();
But this is not: Link(string).filter().split(" ").each();
Title: Re: Link.js v0.2.9
Post by: Fat Cerberus on March 17, 2014, 10:13:21 pm
Very interesting: I just checked the Windows event viewer after Spectacles crashed again, and it's a segfault.  That's not the interesting part, though, THIS is:
Quote
Faulting application name: engine.exe, version: 1.5.0.0, time stamp: 0x487de926
Faulting module name: unknown, version: 0.0.0.0, time stamp: 0x00000000
Exception code: 0xc0000005
Fault offset: 0x00000000
Faulting process id: 0x1940
Faulting application start time: 0x01cf424eb43b96a6
Faulting application path: C:\Sphere1.5\engine.exe
Faulting module path: unknown
Report Id: f3a6c5b5-ae41-11e3-826c-a088692e88a2
Faulting package full name:
Faulting package-relative application ID:


...it's a null pointer dereference.  Now I'm not sure what's causing the issue, as I'm pretty sure I may have dug up an honest-to-goodness, legitimate engine bug.
Title: Re: Link.js v0.2.9
Post by: Flying Jester on March 17, 2014, 10:51:07 pm
I'd be interested in seeing a stack trace of this. Bear in mind, I've found bugs in libJS 1.5 before, so it might not really be fixable.
Title: Re: Link.js v0.2.9
Post by: Fat Cerberus on March 17, 2014, 11:23:02 pm
The crash report in the event viewer doesn't have a stack trace.  Would I be able to get one by running the engine under the VS debugger or do I need a debug build of Sphere for it to be useful?  Because I don't have a copy of VC++ 6.0 handy anymore to do that...
Title: Re: Link.js v0.2.9
Post by: Rahkiin on March 18, 2014, 12:15:50 am
A true release build has stripped symbols, and its stacktrace is completely useless, unless VC has symbolification like Xcode does and a symbol file is available.
Title: Re: Link.js v0.2.9
Post by: Fat Cerberus on March 18, 2014, 01:51:51 am
By default, VC does generate symbol files (.pdb) even in release mode--to make diagnosing release mode-only bugs easier I assume--but they aren't included with Sphere at any rate.  Oh well.
Title: Re: Link.js v0.2.10
Post by: Radnen on March 19, 2014, 03:20:12 am
Bruce, FatCerberus, LordEnglish, PowerCommand, whatever you are: Link 0.2.10 might now fix your Sphere 1.5 issues. If not, it helps in SSFML anyways. :)
Title: Re: Link.js v0.2.10
Post by: Fat Cerberus on March 27, 2014, 12:19:12 am
Haven't seen any segfaults since upgrading to 0.2.10, so I guess that was it.  What was the exact issue, anyway?  You said it was a severe bug, so I was curious why it was so severe.

Oh, new idea for aliases: 'include' and 'exclude' for filter and reject, respectively.
Title: Re: Link.js v0.2.11
Post by: Radnen on March 27, 2014, 01:55:11 am
It was severe 'cause it exploited perhaps a bug of spidermonkey.

The line of code I fixed went like this
Code: (javascript) [Select]

// from:
var args = [].splice.call(arguments, 0, arguments.length);

// to:
var args = [].slice.call(arguments, 0);


That was shorthand for remove nothing, append the current length. Which was weird. The arguments then get passed on and would work without issue... until you run out of memory. I don't know how but by doing the above the arguments grow out of proportion and I guess that memory isn't handled right in SM 1.5, hence the immediate crash after a while of use. Slice makes sure to do a proper cast to an array, adding nothing extraneous to the array. It was a weird mistake on my end, because I'm stupid and didn't know what splice did, even though it seemed to work in testing.

include and exclude... hmmm... ok. Aliases are cheap to make in Link. :) I might make an alias factory for link... that solves all alias problems if I offload them to the end user:

Code: (javascript) [Select]

Link.alias('filter', 'include').alias('reject', 'exclude');


I think it's for the best. I've noticed through my research of many of these kinds of libraries that they all use slightly different naming conventions and conventions seem to change over time.

Edit:
I updated Link to 0.2.11 which see the newly added alias method.
Title: Re: Link.js v0.2.11
Post by: Fat Cerberus on March 27, 2014, 03:04:44 am
Yay, now I can make my own aliases!  Actually I probably won't use it, but I can see other people finding a use for it.

I'm finding Link to be invaluable for Spectacles development.  I've been using it a lot in the implementation of my complex statuses with all their special conditional clauses. :-)
Title: Re: Link.js v0.2.11
Post by: Fat Cerberus on April 15, 2014, 09:17:26 pm
New feature idea, assuming this can't already be done somehow: Some sort of feature where you can query for multiple different items at once.  For example, I have a tagging feature for statuses where they can be tagged with such things as "ailment", "debuff", etc.  There doesn't seem to be a simple way to run a query that checks for multiple tags at once.  Here's an example of something I'm currently doing with Link:

Code: (javascript) [Select]
BattleUnit.prototype.liftStatusTag = function(tag)
{
var statusIDs = Link(this.statuses)
.where(function(status) { return Link(status.statusDef.tags).contains(tag); })
.pluck('statusID')
.toArray();
for (var i = 0; i < statusIDs.length; ++i) {
this.liftStatus(statusIDs[i]);
}
};


For reference, a status definition in Specs looks like this:
Code: (javascript) [Select]
disarray: {
  name: "Disarray",
  tags: [ 'acute' ],
  initialize: function(unit) {
    this.actionsTaken = 0;
  },
  acting: function(unit, eventData) {
    if (eventData.action.rank != Infinity) {
      eventData.action.rank = Math.floor(Math.min(Math.random() * 5 + 1, 5));
    }
    ++this.actionsTaken;
    if (this.actionsTaken >= 3) {
      unit.liftStatus('disarray');
    }
  }
},


Basically what I want to be able to do is query a unit's list of active statuses, and then filter the result based on whether any status's tags array contains any tags listed in ANOTHER array (example: querying for both debuffs and ailments), and then operating on the result (to continue the example, lifting those statuses returned by the query).

I guess essentially what I'm asking for is a vector version of .filterBy(), now that I think about it.  filterBy takes the name of a scalar member and passes any item for which that member's value matches one specified, whereas this would take the name of an array member and pass any item for which that array contained any value in a specified array.  e.g.:
Code: (javascript) [Select]
Link(statuses).vectorFilterBy('tags', [ 'ailment', 'debuff' ]).each(function(item) {
    // ...
}


Maybe even have a hybrid scalar/vector version as well, that checks whether a scalar member matches any of a list of values.
Title: Re: Link.js v0.2.11
Post by: Radnen on April 15, 2014, 09:25:14 pm
I see you want to check against a list of items in an or-like fashion. Currently or-like behavior is not present in a few of the operations. The chains are and-like where if they fail the query is terminated.

I see. I'll look into this and add the functionality.

Edit: ok, I made an ultra fast version of filterBy, but it seems that it's fast because I rely on the array.indexOf feature which is not available in SM 1.5. Otherwise it's fairly slow (I'd be nesting a Link.contains in the filter, which makes it much slower than the array.indexOf method).
Title: Re: Link.js v0.2.11
Post by: Fat Cerberus on April 16, 2014, 01:48:19 am
Is it still just as slow if you use a polyfill for indexOf()?
Title: Re: Link.js v0.2.11
Post by: Radnen on April 16, 2014, 07:01:01 pm
It's twice as slow with a polyfill and I suspect in Sphere it'd be even slower. It was fast because the array's indexOf function is a native method and so gets all the benefits of a purely compiled language like C.

But, with a simple hand-written indexOf it seems to do well, really well! So I'll just do this. I'll update the code in a bit.

Here is the api (so it's non-breaking):
Link(array).filterBy('name', itemA, itemB, ...);

But, this is faster than filterBy by 7 times:
Code: (javascript) [Select]

Link(array).filter(function(item) {
   return item.name == itemA || item.name == itemB;
});


So, if you truly want speed a custom filter is always better. I still want to add in the filterBy because at least it's still more modular than having to write different filters for each test.

I'm changing the api again. It's now like this:
Link(array).filterBy("name", A);  // a single entity
Link(array).filterBy("name", [A, B, C, ...]); // an array of items

In this way it's obvious the algorithm is optimized in the case when you are filtering by a single item. This alone is still much faster than a pluck-then-filter. I wanted to also go with an array so it can be modified elsewhere before introducing it to the function call. I realized a multiple-arguments approach reduces the way in which you can put in inputs to the function.
Title: Re: Link.js v0.2.11
Post by: Fat Cerberus on April 17, 2014, 12:59:33 am
That's a good start, although it doesn't fully solve my tags problem.  My statuses are defined with an array of tags, what I want is to be able to query a unit's list of statuses and return the ones whose taglist contains any of the specified values.  Basically an or-like filter in 2 dimensions, or if you will, vector filterBy().  I do realize this is potentially very slow (if I'm figuring this right it's roughly O(n^3) (cubic) complexity in the worst case), but I wouldn't be doing it ridiculously often either.  Maybe once on a battler's turn, to heal a bunch of similar statuses at once, for example.

I can't even do .expandInto('tags') since that remaps the whole query.  Honestly I'm not even sure I could implement this with a custom filter(), although maybe I'm just not using my imagination enough...
Title: Re: Link.js v0.2.11
Post by: Radnen on April 17, 2014, 01:40:56 am
What about this? All functions contain two parameters, the filtered result and the original 'parent'. Like this:

Code: (javascript) [Select]

Link(array).pluck("prop").each(function(item, parent) { /* ... */ });

// example of printing hello world 3 times from use of pluck:
var item = { greet: "hello", object: "world" };
Link([item, item, item]).pluck("greet").each(function(item, parent) { print(item + " " + parent.object); });


Although the query continues, the second param has the original host object. I don't know how this'll affect speed though...
Title: Re: Link.js v0.2.11
Post by: Fat Cerberus on April 17, 2014, 02:23:52 am
That's a nice idea, I'm not sure how this solves my tag-search issue though.

You understand what I'm trying to accomplish, right?
Title: Re: Link.js v0.2.11
Post by: Radnen on April 17, 2014, 03:31:20 am

You understand what I'm trying to accomplish, right?


Oooh, I see now. Tags is the array and you are wanting to check many items against one without resorting to another Link context. You basically want a contains that conditionally continues the chain rather than stops it.

Code: (javascript) [Select]

BattleUnit.prototype.liftStatusTag = function(tag)
{
        var me = this; // to prevent use of a .bind();
        var statusIDs = Link(this.statuses)
                .pluck('statusDef') /* problem area here, we lose the original status array */
                .hasIn('tags', tag) /* or whatever */
                .each(function(statusDef, status) { me.liftStatus(status.statusID) }); /* second param supplied by pluck */
};


I think that may do what you want per your example. But I don't have a hasIn() method implemented yet, so I'll go do that. Also this would make use of the second parameter containing the parent object belonging to the property of that initial pluck since pluck can whittle down the size of the data being processed.

This would take a lot of existing reworking and I'm not sure what the payoff is (the pluck adding parent part). Is this closer to what you want?

Edit: Option 2 doesn't solve the problem by reducing total number of link contexts, but I think it's a bit more elegant. I don't think this is a bad idea at all, in fact the use of multiple link contexts can help with readability and what you are trying to do in an inner-loop. The operation here is around O(m*n) in complexity, where m is number of statuses and n is number of tags.
Code: (javascript) [Select]

BattleUnit.prototype.liftStatusTag = function(tag)
{
        var me = this; // to prevent use of a .bind();
        Link(this.statuses).each(function(status) {
                if (Link(status.statusDef.tags).contains(tag)) me.liftStatus(status.statusID);
        });
};
Title: Re: Link.js v0.2.11
Post by: Fat Cerberus on April 17, 2014, 10:36:26 am
Okay, we're getting closer, but what I actually want is this:

Code: (javascript) [Select]
BattleUnit.prototype.liftStatusTags = function(tags)
{
    var me = this; // to prevent use of a .bind();
    Link(this.statuses).each(function(status) {
        if (Link(status.statusDef.tags).containsAny(tags)) me.liftStatus(status.statusID);
    });
};


...except ideally there should also be a filter equivalent of containsAny as well.  "OR-like filter in 2 dimensions" is the best description I can give for it.  Which, yes, I realize, is O(m*n*l) complexity.
Title: Re: Link.js v0.2.11
Post by: Radnen on April 17, 2014, 05:15:19 pm
Many to many filtering. I see. I can do this for contains, but not for filter, or otherwise unroll becomes a weird thing. Like why unroll when you can do a 2D filter? Hmm... But then again it's all about keeping the original context.

I still don't know how a 2D filter can help you there, namely because of checking inside of statusDef. The only sensible way I can think of is schema-based filtering, like so:

Code: (javascript) [Select]

Link(this.statuses).orFilter({ statusDef: "tags" }, [tagA, tagB, ...]).each(function(status) { me.liftStatus(status.statusID); });

// alternatively:
Link(this.statuses).orFilter("statusDef.tags", [tagA, tagB, ...]).each(function(status) { me.liftStatus(status.statusID); });


But I'm sure the speed will take a small hit each time to parse the filter condition. This might just be it, then.
Title: Re: Link.js v0.2.11
Post by: Fat Cerberus on April 17, 2014, 05:25:49 pm
If all you can do is containsAny, then that's fine-- it would serve my needs.  I can definitely understand not wanting to do the 2D filter as it would be a major performance hit if abused.  The hit is forgivable with a one-time check like contains, not so much in a general-purpose filtering operation. :)
Title: Re: Link.js v0.2.12
Post by: Radnen on April 17, 2014, 07:46:16 pm
Ok, I released v0.2.12 that adds .has(prop_name, array_or_value), which does basically an eager filter on the inner contents of an array of an object.

it also adds array options to filterBy(prop, values) and contains(values). There is no containsAny, instead Link(stuff).contains([...]) will assume to do an or-like contains any on the data. I don't see much use of an and-only contains, but if I were to do that I could just use a predicate instead.
Title: Re: Link.js v0.2.12
Post by: Fat Cerberus on April 18, 2014, 01:10:34 pm
I don't know, containsAny seems clearer.  I know I'm just nitpicking at this point, but if I see a bare .contains() being passed an array, I would intuitively expect it to check whether it contains all the values--all of them--not only some of them.  I suppose using .some() makes things clearer, but...

Anyway, with 0.2.12, this would do what I want, correct?:
Code: (javascript) [Select]
BattleUnit.prototype.liftStatusTags = function(tags)
{
var me = this;
var statusIDs = Link(this.statuses)
.where(function(status) { return Link(status.statusDef.tags).some(tags); })
.pluck('statusID')
.each(function(statusID)
{
me.liftStatus(statusID);
}
};


Edit: Just tested it, it doesn't appear to be working--the query is returning no results even when there are matching statuses in effect.  Am I doing something wrong (the query looks fine at any rate), or is Link bugged?
Title: Re: Link.js v0.2.12
Post by: Radnen on April 18, 2014, 03:10:11 pm
That should work... My tests for contains seem to pass. I should do proper unit tests, but I figure if my manual tests pass it should work in many circumstances.

Ok... I tested this minial version in both Chrome and Sphere and you'll be shocked. It doesn't work in Sphere.

Code: (javascript) [Select]

var statuses = [
{ id: 0, statDef: { tags: ["heal", "death", "burn"] } },
{ id: 1, statDef: { tags: ["heal", "spark"] } },
{ id: 2, statDef: { tags: ["drain", "turn"] } }
];

Link(statuses).where(function(status) { return Link(status.statDef.tags).some(["burn", "drain"]); }).each(function(status) {
console.log(status.id); // I get 0 and 2 in chrome; nothing in Sphere
});


However if the array has 1 item in it, it works in both chrome and Sphere. I don't know exactly how to solve this since there are no runtime errors. :/

Edit:
Nevermind, it works in both. I forgot to update the version. I honestly don't know why it doesn't work for you. Try running the code above, it should work no different.
Title: Re: Link.js v0.2.12
Post by: Fat Cerberus on April 18, 2014, 03:43:34 pm
Yeah, it was stupidity on my part.  I found out I had a bad item definition which was causing liftStatusTags to be passed undefined instead of the array it's supposed to get.  I feel embarrassed now. :(

Also I love how your little example alludes to a status tagged both 'heal' and 'death'.  Consider its other tag is 'burn', I can't even imagine what kind of unholy status ailment that's supposed to represent. :P
Title: Re: Link.js v0.2.12
Post by: Fat Cerberus on April 19, 2014, 01:01:23 am
Hey, what value exactly do methods like .first() return in case no matching item is found?  It must be something other than null or undefined, because my engine is choking when an AI battler attempts to target a unit by name that doesn't exist, despite a null check on the search result.
Title: Re: Link.js v0.2.12
Post by: Radnen on April 19, 2014, 01:38:45 am

Also I love how your little example alludes to a status tagged both 'heal' and 'death'.  Consider its other tag is 'burn', I can't even imagine what kind of unholy status ailment that's supposed to represent. :P


In Morrowind and Oblivion - when you could make spells - I did all kinds of strange masochistic stuff like spells that burn and heal at the same time. :P


Hey, what value exactly do methods like .first() return in case no matching item is found?  It must be something other than null or undefined, because my engine is choking when an AI battler attempts to target a unit by name that doesn't exist, despite a null check on the search result.


First() returns undefined if nothing can be returned; a single item if only a single item was found, and an array if more than one item was found. I've seen it implemented in this way for libraries like Lazy.js, but in practice I honestly can't say how well that works. I hope it isn't returnning an empty array! :o
Title: Re: Link.js v0.2.12
Post by: Fat Cerberus on April 19, 2014, 03:21:47 am
Looks like first was returning undefined after all--it's just that I was testing against null via strict equality (===), which obviously doesn't match undefined, and so it passed through the undefined as the target of the attack.  This caused the engine to choke when executing the attack since it was trying to run methods on undefined (which, um... doesn't have any, as it turns out).  Oops!

Thanks for the heads-up on the possibility of returning an array, though.  That would have been a nasty bug to run into down the line--the way I implemented attack targeting in Specs, I'd have single-target attacks hitting multiple units and not know what caused it!
Title: Re: Link.js v0.2.12
Post by: Radnen on April 25, 2014, 02:41:37 pm
I notice you have lines like this in your code:

Code: (javascript) [Select]

liftStatusTags: function(actor, targets, effect) {
for (var i = 0; i < targets.length; ++i) {
targets[i].liftStatusTags(effect.tags);
}
},


I think I shall add the ability to pass parameters to .invoke:

Code: (javascript) [Select]

liftStatusTags: function(actor, targets, effect) {
Link(targets).invoke("liftStatusTags", effect.tags);
},


So, what do you think? Again, bear in mind won't be as fast due to some overhead.
Title: Re: Link.js v0.2.12
Post by: Fat Cerberus on April 25, 2014, 03:07:02 pm
I was actually about to suggest that the next time I was around a computer - there was one situation earlier where I wished for the ability to pass arguments to invoke, I even checked the Link code to see if it was already possible!  Sadly it was not...
Title: Re: Link.js v0.2.12
Post by: Radnen on April 25, 2014, 07:44:27 pm
Ok, it's updated :)
Title: Re: Link.js v0.2.12
Post by: Fat Cerberus on April 25, 2014, 10:27:06 pm
Just out of curiosity, how much performance overhead does a bare Link().each have over a for loop?  I often refrain from using Link if there is no filtering needed, but Link.each is definitely more elegant than the ugly C-derived for syntax, so I'm wondering if the tradeoff is worth it.
Title: Re: Link.js v0.2.12
Post by: Radnen on April 26, 2014, 12:55:30 am
I had a long post here, but I kept growing it... And it was a lot of talk. Here's what I'm saying more elegantly:

Link.each is slower than a raw for loop by a small degree, more-so under SSFML.

Creation of Link contexts has no real effect on speed. ~.0001%

Embedding functions has no real effect on speed. ~.0001% (I'm guessing SpiderMonkey caches those functions for you).

Use Link on complex queries, and Link.each to run loops in non-critical sections of your code. Under the chrome browser, well heck, none of this matters because everything is so much faster.
Title: Re: Link.js v0.2.12
Post by: Fat Cerberus on April 26, 2014, 08:37:37 am
So then TurboSphere would be the same deal as Chrome speed-wise?  Since both use V8...
Title: Re: Link.js v0.2.12
Post by: Radnen on April 26, 2014, 12:25:09 pm

So then TurboSphere would be the same deal as Chrome speed-wise?  Since both use V8...


Yeah, it should perform very well under TS, technically.
Title: Re: Link.js v0.2.12
Post by: Flying Jester on April 26, 2014, 09:40:06 pm
I imagine if you used a slower engine-supplied construct it could slow it down.

Otherwise, it should be very similar.
Title: Re: Link.js v0.2.12
Post by: Fat Cerberus on April 27, 2014, 12:04:51 pm
Hm, Link.invoke() with arguments is broken.  It passes the function itself as this instead of the object it's called on.  I ended up breaking my whole Stance system because of this, it took me a half hour to figure out what happened! :'(
Title: Re: Link.js v0.2.12
Post by: Radnen on April 27, 2014, 12:21:21 pm
I don't know why here is where the args are passed into the invocation:

Code: (javascript) [Select]

InvokeArgsPoint.prototype.exec = function(item) { item[this.name].apply(item[this.name], this.args); }

InvokeArgsPoint.prototype.run = function(a) {
var i = 0, l = a.length, n = this.name, args = this.args;
while(i < l) { var m = a[i++][n]; m.apply(m, args); }
}


In both cases I clearly pass the object to it. Wait. I see the issue. I think I'm passing the function to itself. Let me fix this, yikes!
Title: Re: Link.js v0.2.12
Post by: Fat Cerberus on April 27, 2014, 12:23:28 pm
I knew I wasn't going nuts!  How I finally discovered the bug was when I added an Abort call on this in one of my invoked functions--it printed out the function body!
Title: Re: Link.js v0.2.12
Post by: Radnen on April 27, 2014, 12:36:48 pm
Fixed. :P
Title: Re: Link.js v0.2.12
Post by: Fat Cerberus on May 07, 2014, 10:03:46 am
So I'm writing the AI for one of my bosses, and I'm wondering, is there a way to do the following in Link?
Code: (javascript) [Select]
var targetID = null;
var maxValue = 0;
for (unitID in this.damageTaken) {
if (this.damageTaken[unitID] > maxValue) {
targetID = unitID;
maxValue = this.damageTaken[unitID];
}
}


I know Link has the max() operation, but if I'm understanding it correctly, that only gives me the value.  What I actually need as output is the unit ID, which is a string--damageTaken is an associative array.
Title: Re: Link.js v0.2.12
Post by: Radnen on May 07, 2014, 10:16:28 pm
Why are you storing damage in an associative array? If damageTaken was an array of { id, value } objects then Link would work perfectly for you.

Code: (javascript) [Select]

var item = Link(this.damageTaken).max(function(item) { return item.value; });
item.id // the answer you are looking for.


I could somehow try and do a basic test to see if you are iterating over an object or a hash map/associative array. It might even be a parameter in Link:
Code: (javascript) [Select]

var item = Link(this.damageTaken, { hash: true }).max(function(item) { return item.value; });
item.key // the answer you are looking for. Notice it's called key now.


Then when it searches this.damageTaken it knows it's a hash map and so 'recasts' it to a { key, value } pair and then continues the chain like before. This way you don't have to change the underlying data and it can iterate over arrays as well as objects.

Actually that may be quite neat. Picture using Link for certain hash maps:
Code: (javascript) [Select]

// updating health of all poisoned players in an associated array: ["Jim" = new Player(), "Bob" = new Player()];
Link(this.players, { hash: true }).pluck("value").has("status", "poison").invoke("hurt", 5, "green");

// or you can do this with a normal array:
Link(this.players).has("status", "poison").invoke("hurt", 5, "green");


It doesn't seem faster but in some situations it can be convenient. Sometimes storing players in a table like the above is a convenient way of keeping track of them and with Link you'll be able to iterate over it.

I'll give it some thought right now. My only concern is such a feature won't play nice with features that have a '.run()' method tied to them since they assume you iterate over an array and hence why they are so fast. I really want to redesign the library into a 0.3.0 version that isn't as bulky. I'll have to narrow down what I can get away with so I don't have to write so much repeated code. It'll likely only work in modern JS environments since I'd make use of newer JS features.
Title: Re: Link.js v0.2.12
Post by: Fat Cerberus on May 08, 2014, 01:24:33 am
This is why:

Code: (javascript) [Select]
HeadlessHorseAI.prototype.onUnitDamaged = function(unit, amount, tags, attacker)
{
if (unit === this.unit && attacker !== null) {
if (!(attacker.id in this.damageTaken)) {
this.damageTaken[attacker.id] = 0;
}
this.damageTaken[attacker.id] += amount;
}
};


If I set it up as an { id, value } table it would be much more work to update, since I would then have to search the table for an existing ID first, and then if it's not found add it.  Here I can almost do it in one step.
Title: Re: Link.js v0.2.12
Post by: Radnen on May 08, 2014, 02:45:19 am
The way I see it you either take the hit here or later on. It depends which gets called more frequently. Quite frankly I don't know what the trade-off is but I do know a for..in in JS can be notoriously slow. So while you do shave time having instant access to damages, you lose it all in the for..in. And without proper benchmarking there is no way to tell which is faster, but my money is on the purely array based approach. I have came to understand that a JS hash table is really best when not being enumerated. Otherwise arrays are great.

Then again you are hardly dealing with 100+ entries, so it shouldn't matter what you choose to use. :P
Title: Re: Link.js v0.2.12
Post by: Fat Cerberus on May 08, 2014, 09:13:37 am
onUnitDamaged is called every time anyone in the battle takes damage (the table is updated only if the Horse is damaged however), while the search is done at most once per one of the Horse's turns--and in the case of the particular attack being targeted, it's not one that's used very often.
Title: Re: Link.js v0.2.12
Post by: Fat Cerberus on June 05, 2014, 11:54:13 am
So um, is Link:random() bugged, or am I doing something wrong?  I have the following query in one of my AIs:

Code: (javascript) [Select]
var combos = Link(Link(this.combos)
    .where(function(combo) { return combo.phase <= this.phase; }.bind(this))
    .sort(function(a, b) { return b.rating - a.rating }))
    .random(2);


The problem: After said query, combo[0] is a valid object, but combo[1] is undefined, leading to a game crash.  The length of the array is indeed 2, as expected.  I notice there's a splice call in Random(), could that be causing it?
Title: Re: Link.js v0.2.12
Post by: Radnen on June 05, 2014, 04:13:49 pm
It could be. Random was carried over from the old query library and made for to run in link. I'll take a look at it when I get home. It is not a lazy implementation of a random function since its hard to go about grabbing a random element from an unknown sized list (due to potential filtering).
Title: Re: Link.js v0.2.12
Post by: Radnen on June 07, 2014, 04:59:22 pm
Ok, I fixed the random thing. But I wanted to add more optimizations and I was able to make the filterBy filtering 6 times faster, from 1.25 seconds to do an intense operation down to 0.20 seconds.

But it's tricky and you need to add some stuff to the query.

Code: (javascript) [Select]

// old way:
var items = Link(array).filterBy("property", value).toArray();

// new way:
function resolve(item) { return item.property  };
var items = Link(array).filterBy(value, resolve).toArray();


How is that faster? Resolving the property you want to filter against in an external function call is a bajillion times better for compilation than it resolving the property like so: item[property], behind the scenes.

Both methods will be accepted in the newer version of link, so you don't have to use the new way. It does take more code. And I want to further explore this so I could write less code.

I have already tried doing Function("a", "return a."+prop);, but it was too slow to construct the resolving function each time the chain was built (still pretty fast, but not faster than the old way).

Now... If I made Link with a schema attached to it... Something like:
Code: (javascript) [Select]

function schema() {
    this.prop = Number;
    this.prop2 = Boolean;
    // ...
}

Link(array, schema).filterBy(a, "prop").toArray();


I could use the schema to resolve the prop (somehow). But I'm not certain on the gains, but at least it will make making assumptions easier since all items in the array expect to implement that schema.
Title: Re: Link.js v0.2.12
Post by: Fat Cerberus on June 07, 2014, 05:19:13 pm
I don't know, I have nothing against optimization in general, but I feel this is making low-level assumptions about the way the compiler works under the hood that could potentially, depending on the specific JS library used, change out from under you and end up making things slower.  Besides the fact that it makes the code more difficult to read--if I'm going to have to pass a function in anyway, I'd rather just go all-in and use a full filter() operation.  I see filterBy() as a convenience, and this takes all the convenience away.

What was wrong with random, by the way?
Title: Re: Link.js v0.2.12
Post by: Radnen on June 07, 2014, 05:22:31 pm

I don't know, I have nothing against optimization in general, but I feel this is making low-level assumptions about the way the compiler works under the hood that could potentially, depending on the specific JS library used, change out from under you and end up making things slower.  Besides the fact that it makes the code more difficult to read--if I'm going to have to pass a function in anyway, I'd rather just go all-in and use a full filter() operation.  I see filterBy() as a convenience, and this takes all the convenience away.


I agree with you there, so I might scrap this new method if I can't make it intuitive enough. I am cheating the compiler (for chrome) by doing this, and it's technically not the best solution (though it is 6x faster). It always seems like more code = more speed, but at more inconvenience which Link tries not to do.

In IE this new method made it almost 7 times faster from 6.49 seconds to 0.86 seconds. But in Firefox it didn't make much of a difference (2x faster), huh. It seems FF has faster property lookup than IE and Chrome, IE being by far the worst.


What was wrong with random, by the way?


I never saw anything terribly wrong with it, but you can ask it to take more than one random item from the array and if that was greater than the array, extra undefined values get thrown in.
Title: Re: Link.js v0.2.12
Post by: Flying Jester on June 07, 2014, 11:16:54 pm
If it's faster in all three of the major JS engines, I think you could say it's just faster as JS.
Title: Re: Link.js v0.2.12
Post by: Fat Cerberus on June 07, 2014, 11:29:41 pm
It's still an extra level of indirection, which may confuse people--if you're passing in a function for this, I can't imagine it's any slower to just use a full filter() referencing the same property.  Like this:

Code: (javascript) [Select]
function filterFunc(item) { return item.prop == "find me"; }
var items = Link(array).filter(filterFunc).toArray();


I imagine that gets you the same benefits (the function is pre-declared and always references the same property so the compiler can optimize it) as Radnen's proposed change to filterBy(), and is clearer to someone skimming over the code than this:
Code: (javascript) [Select]
function resolve(item) { return item.prop; }
var items = Link(array).filterBy("find me", resolve).toArray();
Title: Re: Link.js v0.2.15
Post by: Radnen on June 08, 2014, 12:02:48 am

It's still an extra level of indirection, which may confuse people--if you're passing in a function for this, I can't imagine it's any slower to just use a full filter() referencing the same property.  Like this:

Code: (javascript) [Select]
function filterFunc(item) { return item.prop == "find me"; }
var items = Link(array).filter(filterFunc).toArray();



Yes, ultimately that is the fastest solution. So in that regard, if you really need the speed for a dire and critical loop, doing just that will indeed speed you up. So I think all bases are covered. I won't go with the resolve approach any longer, but I think it was still a neat exercise in how JS can still be made fundamentally faster by doing little tricks like that.

(btw v0.2.15 is out)
Title: Re: Link.js v0.2.15
Post by: Fat Cerberus on June 08, 2014, 12:13:03 am
So just to be clear, random() still won't choose the same item more than once, correct?  So if I request 2 items and there's only one that matches the query, I'd only get an array of length 1 as the result?  Hm, looks like I will have to keep doing it the way I'm currently doing it for this particular case, but good to know I won't get an array with undefined entries anymore anyway! :-)
Title: Re: Link.js v0.2.15
Post by: Radnen on June 08, 2014, 12:15:29 am

So just to be clear, random() still won't choose the same item more than once, correct?  So if I request 2 items and there's only one that matches the query, I'd only get an array of length 1 as the result?  Hm, looks like I will have to keep doing it the way I'm currently doing it for this particular case, but good to know I won't get an array with undefined entries anymore anyway! :-)


Yeah, it was made to take unique random samples. Uhh, hmmm... I could add a unique argument to it if you want so you can toggle between not unique and unique, it wouldn't be too much to add.
Title: Re: Link.js v0.2.15
Post by: Fat Cerberus on June 08, 2014, 12:34:11 am
Yeah, my use case is this: The boss's AI picks two random move chains from a list of them, like this:

Code: (javascript) [Select]
var combos = Link(Link(this.combos)
    .where(function(combo) { return this.phase >= combo.phase; }.bind(this))
    .random(2))
    .sort(function(a, b) { return b.rating - a.rating; });


And uses them against the two PC battlers, the less powerful one being used against one battler as a distraction while the "real" tactic is being set up under the player's nose.  The thing is, there's no reason the two combos picked have to be unique--the boss could very well choose to use the same tactic against both.  Right now though, to allow this, I have to invoke the RNG myself, and then manually test which of the two tactics chosen has the higher rating--something the sort would do for me, besides being more scalable if at some point I decide to have more than 2 PCs.

How about this: Make random() and sample() distinct.  The former would pick random items without a uniqueness check, while sample() would give unique samples.  Since they're separate use cases, I think this is a better solution than taking an additional parameter.
Title: Re: Link.js v0.2.15
Post by: Radnen on June 08, 2014, 12:55:01 am
Sure, I like that a lot. Random = multiple returns, while sample = unique returns.

I have just now added concat. But this is a lazy concat. Picture doing this:
Code: (javascript) [Select]

Link([1, 2, 3]).concat([1, ..., 1000000]).take(5).toArray();


In a traditional style, it'll concat the first 3 items to the other million items then strip out and return the first 5. In Link it will say, "oh run the query and add the first 3 items, then two more and quit". It should be really fast. :)

Edit: ok, it is done. I'm still calling this v0.2.15 until I have ran out of ideas, then I'll push it officially as a release.
Title: Re: Link.js v0.2.15
Post by: Fat Cerberus on June 08, 2014, 01:07:40 am
That's awesome.  I can't imagine any use cases for it yet, but I'm slowly finding uses for many Link functions I originally thought useless as I've been programming my boss AIs, so who knows. :-)

Besides core game engine stuff, Link seems to be an awesome tool for writing RPG battler AIs--it allows me to simplify so many complex checks and decisions made by the AI into a line or two that would otherwise end up as a massive tower of if conditions or similar.

edit: Just looked at the commit for lazy-concat, looks like you also allow Link contexts to be concatenated as well.  That's a nice touch. :D  Although you may want to note that such feature exists in the documentation...
Title: Re: Link.js v0.2.15
Post by: Radnen on June 08, 2014, 02:33:26 am
I'm planning on adding more things over to Link contexts, so as to make it seamless.

The documentation is really going to be something that will take time. I haven't yet made any formal documentation and am totally planning to do this at some point of time. The sooner the better.

I want to get my own webspace sometime within the week, then make myself a personal website and put the link documentation there. That's the overarching plan.

I'm going to add join now.
Title: Re: Link.js v0.2.15
Post by: Fat Cerberus on June 08, 2014, 02:40:12 am
Are Link contexts immutable, or no?  For example, if I do:

var q = Link(array).filterBy('eatenness', 812)

Can I later re-use q with different operations, or will that mutate the context and cause issues?  This is something I've been wondering about for a while.

Oh, and I know there's no formal docs yet, I just meant in the readme, there's no indication that concat will accept another Link context in place of an array.
Title: Re: Link.js v0.2.15
Post by: Radnen on June 08, 2014, 03:35:46 am
Yes, you can resuse q with different operations. The contexts are immutable as of v0.2.6, so there won't be issues if you decide to store a partial chain, then tack stuff on to it. In fact it's really efficient that way!

I'll try updating more docs as I add more Link integration in its own methods.

I was able to add split, finally. It's not flawless in fact it relies on me appending a "\0" to the end of a string. It sucks, I know, but there's no way of telling whether or not you reached the end of a string in JS, and there's no way of telling wther you are on the last element of a Lazy-executed chain. It works for my small battery of tests on it, so I think it's okay to do so. Though I wonder what the impact is to innocuously add a new element to the end of a string. Otherwise I can't see any way to do this without first resolving the end of a chain by running it, then splitting, then continuing in an entirely new link context. (Which is not necessarily lazy execution by any means). So this shall suffice.

I also added Join. Now there was an SQL-like join already in it, but this is an array join where it returns a string delimited by the character you choose to use.

Join is not lazy, because it returns a compiled instance: the string of the contents joined together. Split on the other hand is lazy and useful for splitting a few elements out of, say, a million.

Say you have a million words separated by spaces but want the first 5:
Code: (javascript) [Select]

var words = Link(long_story).split(" ").take(5).toArray();


I see uses for this in dialog scripts all over the place. And it's fast too since the entire conversation doesn't need to be split, just the first few words or a section of words you decide to choose.

Split cannot detect the future, though, so:
Code: (javascript) [Select]

var words = Link(story).split(" ").skip(5).take(5).toArray();

Means 10 words still need to be split, but no more.

...I could implement a new splitSkip() method that will skip the first n occurrences of a delimiter, but that's the best it gets for Link.
Title: Re: Link.js v0.2.15
Post by: Fat Cerberus on June 08, 2014, 04:27:07 pm
JS strings have a length property, just like arrays.

Code: (javascript) [Select]
Abort("Scott Starcross".length);

...prints 15.  Can't you use that to determine when you reached the end instead of appending nulls?
Title: Re: Link.js v0.2.15
Post by: Radnen on June 08, 2014, 04:37:39 pm
Because you can filter out stuff.

Code: (javascript) [Select]

return Link("Scott Starcross").reject(["s", "t", "a", "r"]).split(" ").toArray(); // ["Sco", "Sco"];


The length will say 15, but the filter now makes it 6, and this is just a simple example. When it reaches the last 'o' it must end after 6 iterations rather than the last s after 15 iterations.
Title: Re: Link.js v0.2.15
Post by: Fat Cerberus on June 08, 2014, 04:47:15 pm
Well in any case, appending nulls shouldn't have any side effects, since JS strings are immutable.  On the flipside, however, that may represent a performance hit if the input string is long enough, since even something as simple as appending a single character means the whole thing has to be copied.
Title: Re: Link.js v0.2.15
Post by: Radnen on June 08, 2014, 05:37:03 pm

Well in any case, appending nulls shouldn't have any side effects, since JS strings are immutable.  On the flipside, however, that may represent a performance hit if the input string is long enough, since even something as simple as appending a single character means the whole thing has to be copied.


Yeah, I was thinking the same thing about how JS appends characters and the performance of doing so. It varies per engine, but from reading stack overflow I think in FF a string appended in such a way is like a linked list in which case the first very long immutable string is kept unaltered in memory and appending something to the end still happens lightning fast since a pointer is generated saying: "wait there's more!". In FF the hit is only taken when reproducing all of the string since all nodes on the linked list must be visited, but in Link.js each character is met one at a time, so unless it has to read the last letter, it shouldn't be much of an issue. I don't know how Chrome works, but I think it's more on the cstring side where memory is reallocated and added when you append characters, but I might be wrong here. V8 keeps changing too fast!
Title: Re: Link.js v0.2.15
Post by: Fat Cerberus on June 08, 2014, 06:24:15 pm
The only other issue I can think of is if the input string already contains '\0' characters to start with, but since the use case here is primarily text, that shouldn't crop up very often, if at all.  We're not parsing binary data after all. :)
Title: Re: Link.js v0.2.15
Post by: Radnen on June 08, 2014, 06:59:40 pm

The only other issue I can think of is if the input string already contains '\0' characters to start with, but since the use case here is primarily text, that shouldn't crop up very often, if at all.  We're not parsing binary data after all. :)


In that case it was meant to prematurely end since '\0' is the standard for the end of text. So if someone does give me that as an issue I'll just say that they need to not use \0 as a delimiter or character in their text, since in C and other languages you should not do that. Still, I don't know if \0 will be a permanent fix, but I'd have to do a lot more retooling to handle the case that the query has ran the "last node". I could do it, but it'd take more code, more complexity, and this really was the best option for now.

At least I can finally do some fast text manipulations in link (which outside of Sphere I am using on various websites).
Title: Re: Link.js v0.2.15
Post by: Fat Cerberus on June 08, 2014, 08:12:19 pm
Could Link be used to tokenize a string?  For example, given the string "812>8", getting back as output [ "812", ">", "8" ].  This is something a normal split can't do, but if you're going for text manipulation as well as array manipulation, it might be a nice feature to have.
Title: Re: Link.js v0.2.15
Post by: Flying Jester on June 08, 2014, 08:41:58 pm
Now you're getting C concerns in JS. Not sure if genius or insanity.

Also, I'm 100% sure there is a standard JS API already there to tokenize strings that is as old as the hills...or at least as old as the SM in Sphere 1.5.
Title: Re: Link.js v0.2.15
Post by: Fat Cerberus on June 08, 2014, 08:44:15 pm
How is tokenization specifically a "C concern"? ???  As for there being a standard API, well, there's a standard API for string split as well...

I guess it is a bit of an edge case, though, and tokenizing a filtered string (Link's strong suit) doesn't really make sense.  Can't seem to find anything about a tokenizer API though, the only thing I can find is String:split, which doesn't tokenize.
Title: Re: Link.js v0.2.15
Post by: Flying Jester on June 08, 2014, 09:14:10 pm
I was referring to worrying about NUL's in strings as a C concern. For C devs, a demon we must all face at some point.

I think it's string.substr, or something like that. It has a somewhat inobvious name.
Title: Re: Link.js v0.2.15
Post by: Fat Cerberus on June 08, 2014, 09:20:51 pm
I was referring to the fact that Radnen is appending NULs to the end of strings internally as a sentinel.  however, that's a valid character to use in a string in JS, so if one is passed in already containing them, it could cause issues.
Title: Re: Link.js v0.2.15
Post by: Radnen on June 08, 2014, 09:22:52 pm
I used Link to tokenize strings for a text-box module where I did some quick styling to letters (such as color, font, style etc.) And the filtering does come in handy if you want code-completion, but I doubt people will do that in Sphere or with the library, but hey, anything's possible.

Yes, it's certainly C-like to use \0 in a string. :P But giving the meaning of \0, I doubt people will casually use it.

Well, a use case for tokenizing I found recently for link under split is the ability to parse urls and file paths. So there's a good example that is often came across in real life.
Title: Re: Link.js v0.2.15
Post by: mezzoEmrys on June 08, 2014, 11:15:40 pm
So you're saying that I shouldn't casually slip null characters into strings I'm planning on processing? Now how am I supposed to hide morse code encoded in null characters!
Title: Re: Link.js v0.2.15
Post by: Fat Cerberus on June 08, 2014, 11:18:33 pm

So you're saying that I shouldn't casually slip null characters into strings I'm planning on processing? Now how am I supposed to hide morse code encoded in null characters!


Haha, good one. :D  But yeah, basically this would only really prevent processing of binary data using Link's string functionality, which is an edge case at best.  Still something to keep in mind, though.
Title: Re: Link.js v0.2.15
Post by: Fat Cerberus on June 23, 2014, 02:34:59 am
So I was about to ask for a new feature earlier, Shuffle, that randomly re-orders the results of the query, another feature I needed for my AI-coding efforts.  But then I did some research and it turns out that not only is the basic Fisher-Yates shuffle very fast at O(n) complexity, but also drop-dead simple to implement.  So I implemented the feature myself. ;D

Radnen, check your GitHub page--there's a pull request waiting.

Usage:
Code: (javascript) [Select]
var spadeDeck = Link(cardPack)
    .filterBy('suit', 'spade')
    .shuffle();
Title: Re: Link.js v0.2.15
Post by: Radnen on June 23, 2014, 09:26:47 am
Love it. Thank you dude!
Title: Re: Link.js v0.2.15
Post by: Fat Cerberus on June 23, 2014, 08:55:02 pm
...and in case anyone is curious what use a shuffle would be for a boss AI, well, here's my use case:

Code: (javascript) [Select]
var targets = Link(this.aic.battle.enemiesOf(this.aic.unit)).shuffle();
var combos = Link(Link(this.combos)
.where(function(combo) { return this.phase >= combo.phase; }.bind(this))
.random(targets.length))
.sort(function(a, b) { return b.rating - a.rating; });
this.tactics = [];
for (var i = 0; i < party.length; ++i) {
this.tactics.push({ moves: combos[i].moves, moveIndex: 0, unit: targets[i] });
}


The AI first asks the battle engine for the identities of all opposing party members, and shuffles that list using Link.  It then picks a number of random move combos equal to the number of opposing party members (@Radnen: This is why I lobbied for random() to be able to return duplicates!), again using a Link query, sorts them by descending rating, and finally builds a tactics table using the results of both queries.  Because the combos are sorted, this guarantees that the first entry in the tactics array is the most powerful, the "primary tactic" so to speak--a guarantee that simplifies other areas of the implementation.  However, it's important that the same unit isn't chosen to be the primary each time, which would end up being the case if not for the shuffle.
Title: Re: Link.js v0.2.15
Post by: Radnen on June 23, 2014, 10:07:12 pm
I'm sure without Link, this would've been a bit harder for you to realize in code what you had in your head. :P

This all sounds great! This is exactly what I made the original Query library for, before it turned into this. :)
Title: Re: Link.js v0.2.15
Post by: Fat Cerberus on June 23, 2014, 10:30:34 pm

I'm sure without Link, this would've been a bit harder for you to realize in code what you had in your head. :P

This all sounds great! This is exactly what I made the original Query library for, before it turned into this. :)


Indeed it would have.  The two main areas where I've found Link to be invaluable are enemy AI and status effects.  Both are cases where the game needs to respond to oftentimes complex conditions while maintaining some semblance of consistent behavior, and without something like Link at my disposal, would quickly devolve into a convoluted tangle of if-elseif blocks, which would be a nightmare to maintain.
Title: Re: Link.js v0.2.15
Post by: Fat Cerberus on July 03, 2014, 09:35:21 am
Okay, maybe I'm just not using my imagination here, but I can't figure this one out.

Suppose I have a dataset like this:

Code: (javascript) [Select]
this.combos = [
{ phase: 1, moves: [ 'electrocute', 'heal' ], rating: 1 },
{ phase: 1, moves: [ 'hellfire', 'windchill' ], rating: 2 },
{ phase: 1, moves: [ 'windchill', 'hellfire' ], rating: 2 },
{ phase: 2, weaponID: 'powerBow', moves: [ 'flareShot', 'chillShot' ], rating: 2 },
// ...and so on
];


And I want to take a number of random items from that array such that:


Is there a way I can construct a single query the meets both criteria?
Title: Re: Link.js v0.2.15
Post by: Radnen on July 03, 2014, 10:05:16 pm
Hmm, it is rather difficult since random must compile the array before executing, but yet you want to strip out multiple occurrences of weaponID after having done that, while maintaining the random count. This is possible if indeed random was not an endpoint but rather a Lazy executed function, of which there are no good algorithms for.

A randomWithFilter is something I'll have to look into.

Edit: for the time being you can add/use this:

Code: (javascript) [Select]

function Random(link, filter, times) {
if (!times) times = 1;
var a = link.toArray();
var samples = [];
while (times > 0) {
var i = Math.floor(Math.random() * a.length);
if (filter(a[i])) {
samples.push(a[i]);
times--;
}
}
return samples;
}


The only issue is that this will loop indefinitely if it does not pass the filter 'times' times. So with the above in mind:
Code: (javascript) [Select]

var found = false, count = 0;
Random(Link(array), function(item) {
    if ("weaponID" in item && !found) { found = true; return true; }
    else {
        count++;
        return item.phase <= phase && count < 6;
    }
}, 5);


It's yucky code and it assumes two things: that the array is at least 5 elements that pass the test, and to pass the test it has to have at least one item with a weaponID and n-1 items that pass the phase check.

The only other option is to programmatically build an array by trying random samples until your requirements are met. It's not so easy to do something like that in a single Link query, but I'll try to think on this some more. It may end up taking n queries best case scenario, and q queries worst case where q = size of list and n = number to receive, where n <= q.
Title: Re: Link.js v0.2.15
Post by: Radnen on July 09, 2014, 09:01:44 pm
Were you able to solve your problem?
Title: Re: Link.js v0.2.15
Post by: Fat Cerberus on July 10, 2014, 02:07:47 am
Sorry, didn't see the edit.  I hate that the notification system doesn't account for edits.  This is why I'll usually make a new post in a case like this--less likely to be missed.

Anyway, I haven't actually touched the Specs code since I asked the question, I'll have to test this out next time I'm on an actual computer and not my phone.
Title: Re: Link.js v0.2.15
Post by: Radnen on November 26, 2014, 05:12:21 pm
"Warning: this topic has not been posted in for at least 120 days."

Lol, anyways,
I thought of a neat feature that would be perfect for link.

Link can be used on objects, it may read their properties (if it doesn't I'll add it) anyways, I run into this from time to time where I'm reading values in from a database and I get string values in return, but want to treat them as numbers. Or I get a text-dump of numbers and I need to convert strings to numbers.

Code: (javascript) [Select]

Link(object).select(['num1', 'num2', 'num3', 'num4']).convert(Number).toObject();
Link(array_of_strings).convert(Number).toArray();


So, what do you think? Neat or not?

... wait I might just get away with:
Code: (javascript) [Select]

Link(array_of_strings).map(Number).toArray();


Which is neat, huh?
Title: Re: Link.js v0.2.15
Post by: N E O on November 27, 2014, 02:30:30 pm
Is it faster to use the built-in Number constructor or to use a custom toInt that essentially returns (n|0)? I haven't jsperf'ed it yet, but if not for the fact that Number is native I'm inclined to think the latter.
Title: Re: Link.js v0.2.15
Post by: Radnen on November 27, 2014, 07:13:16 pm
Here's a link to the test cases.
http://jsperf.com/link-number-conversion

Number seems to win. But feel free to play around with it.
Title: Re: Link.js v0.2.15
Post by: Flying Jester on November 27, 2014, 07:32:12 pm
http://jsperf.com/link-number-conversion/2 (http://jsperf.com/link-number-conversion/2)

In Firefox at least, n<<0 seems to win (even over n>>0). Which makes sense to me, since I thought that's the JSM way of doing it.
Title: Re: Link.js v0.2.15
Post by: Fat Cerberus on November 27, 2014, 07:34:54 pm
Tested it in android firefox (GS5), 3.38 ops/s for Number vs. 3.61 ops/s for n|0.

Edit: Just tested on Safari on my G1 iPad mini: The tests took forever, but Number won by a large margin--0.86 ops/s while everything else was in the 0.6-0.7 ops/s range.  Apparently which method is fastest is browser-specific.
Title: Re: Link.js v0.2.15
Post by: Radnen on November 28, 2014, 01:32:54 pm

http://jsperf.com/link-number-conversion/2 (http://jsperf.com/link-number-conversion/2)

In Firefox at least, n<<0 seems to win (even over n>>0). Which makes sense to me, since I thought that's the JSM way of doing it.



In Chrome, Number still beats everything. :o
Title: Re: Link.js v0.2.15
Post by: Fat Cerberus on November 28, 2014, 01:39:17 pm
Yeah, Number beat everything on iOS Safari as well, as mentioned above.
Title: Re: Link.js v0.2.15
Post by: N E O on November 28, 2014, 02:23:17 pm
http://jsperf.com/link-number-conversion/3 (http://jsperf.com/link-number-conversion/3)

What happens when you don't use anonymous functions in the loop. Number is the slowest of the bunch, but I'm only using OS X Safari 7.1. I'm pretty sure I'd get the same results as you guys if I were using other browsers.
Title: Re: Link.js v0.2.15
Post by: Radnen on November 28, 2014, 09:02:33 pm

What happens when you don't use anonymous functions in the loop.


In Chrome there are no changes. I think most JIT's will cache anonymous functions, and besides an initial speed hit, subsequent calls to the function wouldn't have much of an impact. It averages out quite nicely. But I guess one bonus of pulling out the anonymous functions is a better understanding to optimize by the compiler. I think you see it most with |0, >>, and << in FF.
Title: Re: Link.js v0.2.15
Post by: Flying Jester on November 28, 2014, 09:48:46 pm
I think you see it most with |0, >>, and << in FF.


I expect a large part of why bitshift-by-zero and other jsm-isms are faster in FF is that Chrome didn't for a long time (and still doesn't?) accept JSM as a thing, so these have special optimizations in FF and not Chrome.
Title: Re: Link.js v0.2.16
Post by: Radnen on February 15, 2015, 01:05:18 pm
Version 0.2.16 has been released. The last official release was 0.2.13, but some of you might have just downloaded 0.2.15 anyways in which case this really only fixes the .toArray() method.
Title: Re: Link.js v0.3.0
Post by: Radnen on February 17, 2015, 12:42:16 pm
So I've put out v0.3.0, it's still in beta since I'm still testing it. I'm upping the minor version count and not the patch count because I added a new feature that changes the behavior of Link. Now it will recognize the array index at every step of the way. This is a minor, minor speed hit but now you have access to an arrays index in each function call:

Code: (javascript) [Select]

Link([1, 2, 3]).map(function(item, i) { return item + i; }).toArray(); // [1, 3, 5] (new array)


Because of this newly added feature I've now added something I've thought of for a very long time. I originally didn't want this behavior because I found it "destructive" but now I think it has a use and so I'm adding it at your own risk. :P

Code: (javascript) [Select]

Link([1, 2, 3]).map(function(item, i) { return item + i; }).fuse(); // [1, 3, 5] (same array)


It'll modify the original array. I also call the feature '.coalesce()' but fuse is probably easier to type and remember. It synergizes well with .pluck:

Code: (javascript) [Select]

Link(array).pluck('apples').map(worms).fuse('apples');


Before this, there was no easy way to access the originating array after a pluck. Now you can pluck something out, do work and fuse it back by using the same property you plucked from, or you can use a different property and store results of the Link expression into another property of the object in the array.

Currently it's only an end point, but I will soon add a non-end point variants so we can jump around the array at will and puck/fuse/pluck other properties at will. This is why the version is at 0.3.0 and I thank you all so far for using it. :)
Title: Re: Link.js v0.3.0
Post by: Fat Cerberus on February 17, 2015, 01:09:04 pm
Awesome, love the index tracking.  I do remember a few instances where it would have been helpful to know the index into the array, so definitely a good addition.

Fuse is neat as well, although I don't think I have a use case for it yet.  I'm sure I'll find one though!  There have been a lot of features in Link that I thought I would never use and ended up using anyway at some point or another.  For instance, random(), which is awesome for enemy AIs.

Just to be sure, is 0.3.0 code-compatible with 0.2.16?  I want to make sure before I just drop the new version into Specs.  That toArray() bug was a nightmare, don't need more Link glitches showing up tricking me into thinking the problem is on my end. ;)
Title: Re: Link.js v0.3.0
Post by: Radnen on February 17, 2015, 02:13:57 pm

Just to be sure, is 0.3.0 code-compatible with 0.2.16?  I want to make sure before I just drop the new version into Specs.  That toArray() bug was a nightmare, don't need more Link glitches showing up tricking me into thinking the problem is on my end. ;)


Well I try to test it in a browser first. That toArray() bug was really, really weird though. I have so far used it as a drop-in replacement of 0.2.16, it really only adds a new parameter to all function calls so unless you are using that space for something else I'm sure it wouldn't hurt. The only bugs I foresee and hence why it is still in beta, is that I might have not added added the index parameter to the chain somewhere. It really only works if that value can be passed on from item to item. For most cases it is working so I think it's 80%+ good.

Oh, also, the index is not the newly filtered index, so be careful, it still references the original index. It does that since you could pluck, filter, and replace into the same indices you plucked from. Getting the filtered or final index should really only be feasible at the end of the chain, so only after a real toArray(). I'm professionally using this library on a few websites I work on, so it has to work (lol), and I can do data conversion really efficiently with it now (creating a new array and copying over the new values is slower than modifying the existing one as you can imagine, it's also impossible to tell where the final values originally mapped without back-peddling until now).

Edit: As a side note, I even added the standard 'undefined' fix to it, just in case :P
Title: Re: Link.js v0.3.0
Post by: Radnen on February 18, 2015, 02:26:27 am
New post, new feature, still v0.3.0:

Code: (javascript) [Select]

var array = [{ a: 1, b: "Hi" }, { a: 2, b: "Hello" }, { a: 3, b: "Bob" }, { a: 4, b: "World" }, { a: 6, b: "skip" }];
console.log(Link(array).pluck("a").filter(even).take(2).unpluck("a").pluck("b").join(" ")); // Hello World


There you go, unpluck. It will remove the need of creating a new link context and prior filters etc. are still honored, so this will result in a speed improvement too.

I also fixed string-based .join() to recognize a default parameter, "" when it is invoked with no parameters. This is the same join that if you pass a function, performs a SQL join, a strange overload but it's semantically sound... I guess.
Title: Re: Link.js v0.3.0
Post by: Radnen on February 18, 2015, 04:52:45 pm
New popst, two new features, still 0.3.0:

Sum() and Average() have been added. I can't believe these had been missing for as long. :/

They both assume the array is numbers by default, but if you pass into them a string property, it'll attempt to use that inside the parent object. Or you can pass a function to either and resolve it dfrom the item argument:

Three ways of doing the same thing:
Code: (javascript) [Select]

Link([{ a: 45 }, { a: 1 }, { a: 12 }]).sum("a"); // 58
Link([{ a: 45 }, { a: 1 }, { a: 12 }]).sum(function(item) { return item.a; }); // 58
Link([{ a: 45 }, { a: 1 }, { a: 12 }]).pluck("a").sum(); // 58
Title: Re: Link.js v0.3.0
Post by: N E O on February 19, 2015, 11:26:04 pm
Sum() and Average() have been added. I can't believe these had been missing for as long. :/


<kidding>I CAN'T BELIEVE YOU'RE STILL MISSING THE REST OF EXCEL'S FUNCTIONS >:(  </kidding>
Title: Re: Link.js v0.3.0
Post by: Fat Cerberus on March 20, 2015, 03:23:40 am
Radnen, thought you might like to know this.  In the most recent duktape release's changelog:

Quote
Fix assignment evaluation order issue which affected expressions like "a[ i] = b[i++]" (GH-118)


This verifies that the Link misbehavior under minisphere was indeed a JS engine bug.  Might want to alert the Jurassic devs now! :P
Title: Re: Link.js v0.3.0
Post by: Fat Cerberus on April 01, 2015, 02:27:56 am
Feature request (unless this already exists): Function to return index in array of first match, like this:

Code: (javascript) [Select]
this.items = [
{ name: "Party", id: 'party' },
{ name: "Items", id: 'items' }
];
this.selection = Link(this.items).pluck('id').indexOf('items'); // == 1


Or something similar.  I can kind of hack a solution together using .toArray(), but that only gives me the index into the *processed* array, whereas I always want it to index against the input array, even if the query filters stuff out.  Is this possible?
Title: Re: Link.js v0.3.0
Post by: Radnen on April 01, 2015, 02:08:03 pm
Did you try that out? It works just fine for me. Are you using v0.3.0?
Title: Re: Link.js v0.3.0
Post by: Fat Cerberus on April 01, 2015, 02:17:24 pm
Huh. So there is an indexOf method in Link after all.  However:

Code: (javascript) [Select]
var items = [
{ name: "Party", id: 'party' },
{ name: "Items", id: 'items' },
{ name: "Battlers", id: 'battlers' }
];
var index = Link(items).filterBy('id', 'battlers').pluck('id').indexOf('battlers'); // == 0, was expecting 2


Is there any way to get it to always return the index in the original array, or no?
Title: Re: Link.js v0.3.0
Post by: Radnen on April 01, 2015, 05:02:37 pm
You could do this:
Code: (javascript) [Select]

var index = Link(items).pluck('id').indexOf('battlers');


If you want to filter for any reason and keep a reference to the original index, there is a way by using the new second argument in link 0.3.0:
Code: (javascript) [Select]

var index = 0;
Link(items).filterBy('id', 'battlers').each(function(item, idx) { if (item.id == 'battlers') index = idx; });
index; // use index from here on out.


Either I need to add a getRealIndex() method (versus the indexOf which gets the post-filtered index) or I research a way to efficiently return early from an each operation.

Then there is the question:
Should indexOf use the post-filtered index or should it grab the original index? I ask because the indexOf operation is fast since I don't have to copy values to a new intermediate array to get the post-filtered index.

It would suck to have to do something like:
Code: (javascript) [Select]

Link(Link(array).filter().toArray()).indexOf();


It's better to:
Code: (javascript) [Select]

Link(array).filter().indexOf();
Link(array).filter().oldIndexOf(); //or whatever I end up calling it


I've always liked the C-style one word naming convention, so I want to avoid an unnecessarily wordy API, even if it describes the action well enough.
Title: Re: Link.js v0.3.0
Post by: Fat Cerberus on April 01, 2015, 05:58:49 pm
Yeah, just a pluck and indexOf will work.  The reason I ask for a version that works even after filtering is mostly hypothetical: suppose you're given a partial chain that already has a few filtering ops attached to it, you can just tack a pluck().indexOf() onto the end of it to still get the original index (assuming it passes the filter anyway).

But yeah, I can see how that might be a performance hit since every link in the chain (...holy crap, the name Link works on so many levels!) will have to track indices anew.  Your choice, just wanted to bring up a potential use case.  For my purposes pluck and indexOf are sufficient.
Title: Re: Link.js v0.3.0
Post by: Radnen on April 01, 2015, 09:46:19 pm
Actually I'm going to totally redefine the behavior of indexOf. It's always going to use the original array for now on. To get the filtered or final index just pass true to the end of indexOf, like so:

Code: (javascript) [Select]

Link([1, 2, 3, 4, 5, 6, 7, 8, 9, 10]).filter(even).indexOf(10, true);// 5


This means removing a feature, indexOf(prop, value). But I think it's okay to remove that feature since you can always use pluck before you call indexOf.
Title: Re: Link.js v0.3.0
Post by: Fat Cerberus on April 01, 2015, 10:27:17 pm
It won't hurt the performance of the other operations if indexOf isn't used will it?  Realistically I don't see indexOf to be a common thing to do with Link, so I'd hate to have something like this bring everything else down.
Title: Re: Link.js v0.3.0
Post by: Radnen on April 02, 2015, 12:46:06 am
Actually I'm making indexOf faster by doing this. The index is already passed as the second parameter I'll just leverage this free gift.

Since this is lazy execution, indexOf makes the entire thing really fast in fact. Imagine only looking as far as the 4th item (because whatever it is, it happens to be the one that matches). Then imagine having 1 million items in the list. It really only executes the chain 4 times, rather than execute everything then find the the item that matches, even if it's only 4th position.

This is why methods like split, indexOf, take, get, contains, every, and first are very fast, they stop as soon as the predicate is matched, after only having executed the chain until then. And in the case of string split, it doesn't have early termination in native JS.

Edit:
I'm going to scrap what I did to indexOf.


suppose you're given a partial chain that already has a few filtering ops attached to it, you can just tack a pluck().indexOf() onto the end of it to still get the original index (assuming it passes the filter anyway).


It shouldn't work like that. Filter dirties the chain. In practice it makes sense for indexOf to return only the new index after filtration. I was thinking maybe not, but this is a lazy execution environment. There are times can no longer get the original index of an item. After some time that data vanishes further into the chain you go.

Take for instance:
Code: (javascript) [Select]

Link([0, 1, 5, 4]).filter(function(n) { return n > 3; }).skip(1).indexOf(5);
Link([0, 1, 5, 4]).filter(function(n) { return n > 3; }).skip(1).indexOf(5, true);


They both return -1 (does not exist), but shouldn't #2 return 2? Well it would if the combination of the filter and the skip doesn't destroy element 5 from existence. See, it only works to get the item in the original array if it survives the chain of operation. So, it makes sense to use the filtered value because in this case at least it's guaranteed to produce a result in alignment with the data it operates on.

But I do see one use of an unfiltered index version of indexOf. There is a speed increase. If you filter out even numbers and you know the item is odd, but you want the original index, and the item is somewhere within the first half of the list. We can filter out the numbers we don't need to check, get to the odd number faster, and then get the index, returning the original index rather than the filtered index. But it might almost be faster to just get the index of the odd number anyways, I'd only see this as a speed increase if you work with millions of numbers. Filter is fast, but indexOf is faster.
Title: Re: Link.js v0.3.0
Post by: Fat Cerberus on May 08, 2015, 09:39:42 am

Edit:
I'm going to scrap what I did to indexOf.


suppose you're given a partial chain that already has a few filtering ops attached to it, you can just tack a pluck().indexOf() onto the end of it to still get the original index (assuming it passes the filter anyway).


It shouldn't work like that. Filter dirties the chain. In practice it makes sense for indexOf to return only the new index after filtration. I was thinking maybe not, but this is a lazy execution environment. There are times can no longer get the original index of an item. After some time that data vanishes further into the chain you go.

Take for instance:
Code: (javascript) [Select]

Link([0, 1, 5, 4]).filter(function(n) { return n > 3; }).skip(1).indexOf(5);
Link([0, 1, 5, 4]).filter(function(n) { return n > 3; }).skip(1).indexOf(5, true);


They both return -1 (does not exist), but shouldn't #2 return 2? Well it would if the combination of the filter and the skip doesn't destroy element 5 from existence. See, it only works to get the item in the original array if it survives the chain of operation. So, it makes sense to use the filtered value because in this case at least it's guaranteed to produce a result in alignment with the data it operates on.

But I do see one use of an unfiltered index version of indexOf. There is a speed increase. If you filter out even numbers and you know the item is odd, but you want the original index, and the item is somewhere within the first half of the list. We can filter out the numbers we don't need to check, get to the odd number faster, and then get the index, returning the original index rather than the filtered index. But it might almost be faster to just get the index of the odd number anyways, I'd only see this as a speed increase if you work with millions of numbers. Filter is fast, but indexOf is faster.


Just noticed this edit (WHY do the forums not add edited posts to the unread list?).

Anyway, I thought that was kind of the point?  If the filtered data doesn't contain the item I ask for, I WANT it to return -1.  Otherwise, if it exists, I want the original index so I can directly index the original array.  I know I had a use case for this, but it's been so long now that I've forgotten what it was.

Also, I found a clever use of Link recently:
Code: (javascript) [Select]
// mini.Console.unregister()
// Unregisters a previously-registered entity.
// Arguments:
//     name: The name of the entity as passed to mini.Console.register().
mini.Console.unregister = function(name)
{
this.commands = mini.Link(this.commands)
.where(function(command) { return command.entity != name; })
.toArray();
};


It can be used to conditionally remove stuff from an array! ;D
Title: Re: Link.js v0.3.0
Post by: Radnen on May 08, 2015, 09:04:12 pm

Just noticed this edit (WHY do the forums not add edited posts to the unread list?).


Sorry I have a bad habit of editing over double-posting when it's been a short time since the last time I posted. I probably should double-post just so it can be seen.


If the filtered data doesn't contain the item I ask for, I WANT it to return -1.


Yeah, I was just thinking of potential side-effects. I'm not too sure if they would cause issues or not. Maybe I was over-thinking.


Also, I found a clever use of Link recently:
Code: (javascript) [Select]
// mini.Console.unregister()
// Unregisters a previously-registered entity.
// Arguments:
//     name: The name of the entity as passed to mini.Console.register().
mini.Console.unregister = function(name)
{
this.commands = mini.Link(this.commands)
.where(function(command) { return command.entity != name; })
.toArray();
};



Cool! I love these approaches in GC'd languages, they are simple, fun and you don't have to worry about garbage arrays piling up. (.toArray() makes a new array each time, but I think that's fine in these cases).

Oh, I've added a recursive mode and other fixes and features to Link, so stay tuned!
Title: Re: Link.js v0.3.0
Post by: Radnen on May 09, 2015, 12:32:42 am
So, new features.

1. .indexOf() now takes a predicate.
Code: (javascript) [Select]
var index = Link(array).indexOf(function(item) { return item.name == "Tom"; })


2. .groupBy() now has a container flag.
Code: (javascript) [Select]
var group = Link(array).groupBy(function(item) { return item.key; }, true); // true = as array


3. recurse() will iterate throughout an array of objects with arrays of objects in them.

Code: (javascript) [Select]

// say you get back the contents of the filesystem, folders with files and subfolders with files.
var structure = [{
    items: [{ items: [...] }, { items: [...] }, ...]
}, {
    items: [...]
}];

// you can easily iterate over and load them using 'recurse'
Link(structure).recurse('items').each(function(item) { LoadFile(item); });


These are still part of v3.0 which is still in beta testing. Recurse itself is still experimental. I've used it in a website that had a hierarchical navigation menu and it worked well enough. Of course more type checking and the like would help, so there's always more to fix.
Title: Re: Link.js v0.3.0
Post by: Fat Cerberus on May 09, 2015, 02:04:24 am
That recursive mode is awesome.  I've actually had a few cases where such a thing would be useful, so good work. :D
Title: Re: Link.js v0.3.0
Post by: Radnen on May 11, 2015, 06:16:39 pm
I hate seeing stray for loops, but there are two areas where they still seem necessary, despite using Link.

A) When you are deleting something:
Code: (javascript) [Select]

for (...) {
    if (condition) { array.splice(i, 1); i--; }
}


B) When you are computing something.
Code: (javascript) [Select]

for (var i = 0; i < 12; ++i) {
    DrawText(0, 16 * i);
}


Case B can be mitigated using lists of items and iterating over them, but I can still create generators for Link to mitigate these cases where you are doing a raw loop 'n' times.

But, most importantly A was counter-intuitive to Link. But I don't think that has to be the case.

Introducing: Remove
I leverage the Link chain and make an end-element such that whatever values survive until the end, can get gobbled up. I think the implementation can be improved upon, but I do utilize a neat trick. I walk the line backwards when removing elements, this works since the indices I receive from Link are in sorted order.

Now you can do this:
Code: (javascript) [Select]

var a = [1, 2, 3, 0, 4, 5, 8, 9, 7, 0];
Link(a).where(even).remove();
a; // [1, 3, 4, 8, 0]


I'm still testing the code, and I'll push it later today. Maybe I can make it faster? But ultimately it's going to make it easier to remove things, such that human induced error goes down.

BTW, this caused me to do it:
Code: (javascript) [Select]

mini.Delegate.prototype.remove = function(o, method)
{
for (var i = 0; i < this.invokeList.length; ++i) {
if (o == this.invokeList[i].o && method == this.invokeList[i].method) {
this.invokeList.splice(i, 1);
break;
}
}
};


Imagine writing:
Code: (javascript) [Select]

mini.Delegate.prototype.remove = function(o, method)
{
mini.Link(this.invokeList).where(function(item) { return o == item.o && method == item.method; }).remove();
};


Wait... I just realized now that you intended to remove the first item it sees (hence the break), well we can now do that with the new functional parameter for .indexOf:

Code: (javascript) [Select]

mini.Delegate.prototype.remove = function(o, method)
{
var index = mini.Link(this.invokeList).indexOf(function(item, i) { return o == item.o && method == item.method; });
if (index >= 0) this.invokeList.splice(index, 1);
};


But that's good for one at a time, whereas now you can do lists of removals (which ought to be faster than looking up indices each time). :)
Title: Re: Link.js v0.3.0
Post by: Fat Cerberus on May 11, 2015, 07:45:46 pm
That delegate code is rather old and was originally written before Link was a thing.  I just didn't bother updating it when I adapted it.  Generally I now know to do search-and-remove like this:

Code: (javascript) [Select]
array = Link(array)
    .where(function(item) { return item != toRemove; })
    .toArray();


So long as it's not important for the array to maintain its identity (and it usually isn't), this works great.  That said, if your .remove() method can remove stuff in-place a la splice(), that's of course even better. :D

Incidentally, do you know how much more awesome Link would be if we would get Fat Arrow support already?  All those "function"s everywhere clutter everything up.  I hadn't noticed it before, until I started working with promises, where you're passing anonymous functions all over the place.  A proper lambda operator would be a lifesaver.
Title: Re: Link.js v0.3.0
Post by: Radnen on May 12, 2015, 12:50:42 pm

That delegate code is rather old and was originally written before Link was a thing.  I just didn't bother updating it when I adapted it.  Generally I now know to do search-and-remove like this:

Code: (javascript) [Select]
array = Link(array)
    .where(function(item) { return item != toRemove; })
    .toArray();



I know, but my solution is a little nicer in the fact we keep the original array reference, it's faster than rebuilding a list (esp. if it's long) and we aern't throwing the old reference away to get gc'd each time. But I will say your approach above is faster on smaller lists. Link was designed for any sized list though, so I was thinking about millions of items when I wrote this.


Incidentally, do you know how much more awesome Link would be if we would get Fat Arrow support already?  All those "function"s everywhere clutter everything up.  I hadn't noticed it before, until I started working with promises, where you're passing anonymous functions all over the place.  A proper lambda operator would be a lifesaver.


I know! It's in ES6, I'd love to see it too. It'll make Link feel much closer to LINQ. :P
Title: Re: Link.js v0.3.0
Post by: Fat Cerberus on May 12, 2015, 10:38:45 pm
Is there any possibility of making .each() chainable?  Such that this:
Code: (javascript) [Select]
mini.Link(this.threads)
.where(function(thread) { return thread.id == threadID })
.each(function(thread) { thread.isValid = false; });

mini.Link(this.threads)
.where(function(thread) { return thread.id == threadID })
.remove();


Could become this:
Code: (javascript) [Select]
mini.Link(this.threads)
.where(function(thread) { return thread.id == threadID })
.each(function(thread) { thread.isValid = false; });
.remove();


The above step (setting thread.isValid to false) is required so that if a thread is killed in the middle of an update cycle (which runs over a copy of the array), it doesn't get erroneously updated again.  But since each() is an endpoint, I have to do it as two operations.
Title: Re: Link.js v0.3.0
Post by: Radnen on May 13, 2015, 01:02:51 am
I like what you propose, I too run into these situations. I was actually thinking about this earlier today at work (funnily enough also due to the addition of remove). The only issue I see here is that as soon as an endpoint is called, the chain gets executed. To get around this I could create a new method or involve a parameter.

These are the options:
Code: (javascript) [Select]

Link(array).execute(function() { /* work */ }).remove();

Link(array).each(function() { /* work */ }, true).remove();


I'll likely implement both, but I'm concerned with the .each because what other use could I use a parameter for? Usually in an each or foreach method in JS the second parameter means to use that context. But we have .bind() for that now. And I don't want to remove a feature after introducing it. So here's what are you okay with .execute()? All methods are Link.alias()-able in the system, so you could rename it.
Title: Re: Link.js v0.3.0
Post by: Fat Cerberus on May 13, 2015, 01:07:16 am
.execute() is fine.  Although maybe .exec() would be more succinct?  I agree about .each() though, the extra parameter is unnecessary and not self-documenting at all.
Title: Re: Link.js v0.3.0
Post by: Radnen on May 13, 2015, 02:47:55 am
Okay, I added .execute().

Also, I made a new branch called no-runners it's a version of Link that strips the runners potion away. It reduces the size of the library and I've since seen no discernable impact in the modern browsers with the runners off.

The runners used to kick-start a loop by suggesting to the compiler that it can inline the first calls of a chain of commands. Instead of going at least one level into the chain to being using where() map(), etc. it would do it right then and there.

Code: (javascript) [Select]

// a runner:
while (...) {
    if (where(a[i], i)) a[i].next.exec(a[i], i);
    i++;
}

// a .exec() method in a node:
this.start.exec();


The runner was more code, but it actually did improve things. Each node that needed kick-starting had a .run() method where you can tell I inline the behavior of the node right inside the logical area of the loop.

Nowadays JIT's have gotten so good, I'm now not noticing much of an impact removing them. So, it's still experimental, but the branch will be the new 0.4.0 once it's ready.

I think I'll release v0.3.0 soon and start supporting it up until I release v0.4.0 which will receive features on it's own.
Title: Re: Link.js v0.3.0
Post by: Fat Cerberus on May 13, 2015, 02:50:32 am
I'd be curious to see a benchmark comparison between the normal and no-runners version under minisphere, just to see if there's any discernible difference on non-JIT engines like Duktape.
Title: Re: Link.js v0.3.0
Post by: Fat Cerberus on May 20, 2015, 10:18:28 am
So, I've been using the new features quite a bit.  Well, except for the recursive support, I haven't managed to shoehorn in a use case for that yet. :P

But .execute() and .remove() are great:
Code: (c) [Select]
var itemCount = 0;
mini.Link(this.items)
.filterBy('itemID', itemID)
.execute(function(usable) { itemCount += usable.usesLeft })
.remove();
if (itemCount > 0)
mini.Console.write(itemCount + "x " + Game.items[itemID].name
+ " deleted from " + this.name + "'s inventory");
else
mini.Console.write("No " + Game.items[itemID].name + " in " + this.name + "'s inventory");

Title: Re: Link.js v0.3.0
Post by: Radnen on May 31, 2015, 04:15:46 am
This is the thickest piece of code I wrote, it's using a new clone method to do a shallow clone of the item in question. I say shallow clone because debatably, a deep clone is really very hard to accomplish in JS (and you could write your own in a map function anyways, besides the best method is that you program a .clone() into the prototype of each object you intend to clone). This shallow clone is more for plain old javascript objects that usually contain raw data (IE: primitive types like numbers, strings, and booleans).

Code: (javascript) [Select]

// create flattened array of items whose sub-items get linked to parent items:
var array = Link($scope.menu).execute(function(item) { item.level = 0; }).recurse("items").clone().map(function(item, idx) {
item.is_parent = item.items.length > 0;
item.index = idx;
Link(item.items).each(function(inner) { inner.level = item.level + 1; });
delete item['items']; // don't need this around anymore.
return item;
}).toArray();


It's for a product menu for a website I'm building. You have categories of items and this flattens the array so it can be stored sequentially into a database. It keeps track of who the parent is and strips away children that don't need to be sent to the database.

It's so dang complex that I delete items from an item. But that's okay since I've used recursion before the clone and since it's a shallow copy, modifications to the inner items array are copied through the next cycle. It's a mind-blowing show-stealing piece of terrible crap that's highly, highly unreadable and can kill at any time if I change the core mechanics of clone or recurse in Link.js so don't try this at home. But I'll still use this since it is so damn efficient.
Title: Re: Link.js v0.3.0
Post by: Fat Cerberus on May 31, 2015, 09:42:33 am
Haha, wow, I couldn't even begin to understand what that code is doing if it weren't for your explanation of it.  If you didn't need to flatten the array, I was going to say, I made my own function a while back for deep-cloning POJOs recursively.  It even preserves circular refs:

Code: (javascript) [Select]
// clone() function
// Creates a deep copy of an object, preserving circular references.
// Arguments:
//     o: The object to clone.
// Returns:
//     The new, cloned object.
function clone(o)
{
var clones = arguments.length >= 2 ? arguments[1] : [];
if (typeof o === 'object' && o !== null) {
for (var i = 0; i < clones.length; ++i) {
if (o === clones[i].original) {
return clones[i].dolly;
}
}
var dolly = o instanceof Array ? []
: 'clone' in o && typeof o.clone === 'function' ? o.clone()
: {};
clones.push({ original: o, dolly: dolly });
if (o instanceof Array || !('clone' in o) || typeof o.clone !== 'function') {
for (var p in o) {
dolly[p] = clone(o[p], clones);
}
}
return dolly;
} else {
return o;
}
}


Notice that if any object in the tree includes a .clone() method, it calls that instead.  It's been quite useful in Specs for copying, e.g. item definitions and such from the gamedef.
Title: Re: Link.js v0.3.0
Post by: Radnen on September 29, 2015, 01:00:16 pm
Casually continuing after many months, I needed the shallow clone in that last post (to keep loose references to child arrays).

Anyways, I've been thinking about cloning and wondered what more I can do to it. I've been thinking about creating a Sphere data storage API. It's interesting because we can use it to standardize save files in Sphere, and not just for games, but for data. Like I use link.js to do a LINQ-to-SQL like implementation in JS for Sphere.

Code: (javascript) [Select]

// set up a database
var enemies = new Link.db({ idfield: "_id" });

// get an id of an item put into it:
var ID = DbLink(enemies).put({ name: "rodent", atk: 5, def: 2 }).getLastId();

// grab that item you added:
var enemy = DbLink(enemies).get({ _id: ID }); // a new instance of an enemy

// But instead of working on objects, you always get clones back.
// Before, where gives you the object. Now, it gives you a copy of the data you stored in the database. To update it, you'll have to do things like:

DbLink(enemies).where({ _id: ID }).update({ name: "rodent2" }); // updates all that match

// might also use?
DbLink(enemies).update({ _id: ID }, { name: "rodent2" }); // matching in the update method

// then we can remove it, completing the CRUD:
DbLink(enemies).where({ _id: ID }).delete();

// or:
DbLink(enemies).delete({ _id: ID });


Notice it's more JSON based than before. I like no-sql like document-driven databases. It may seem like reinventing the wheel for those who use MongoDB or CouchDB etc. But I think we can make a Sphere centric DB and I get to use more JSON in my Link library. I'll need to think a bit more about the API, but I see this as a natural next step. This will of course, amend Link.js as a separate library (or not, I mean get is the same as where, except that you get a copy, it can fit in nicely with the overall structure, and there was no way to put something into it before).

I've called it DbLink as a test, it may just amend Link, nothing more.
Title: Re: Link.js v0.3.0
Post by: Fat Cerberus on September 30, 2015, 01:02:17 pm
I think if we were going to standardize on a save file format (as part of Sphere 2.0 perhaps), JSON would be a no-brainer.  It loads directly into a POJO and all modern JS engines include a parser built-in.

The clone-based approach is nice.  I kind of do something similar with Specs, where battlers and such are constructed by cloning a descriptor from the gamedef, so modifications don't touch the original definition.  Having that in a more general capacity would be useful, I think.  Although in my case, I need a deep clone (see cycle-preserving clone function above), since the descriptors have a lot of nested stuff.  Moves have a list of turns, which have a list of actions, etc.  That kind of thing.
Title: Re: Link.js v0.3.0
Post by: Fat Cerberus on October 16, 2015, 02:53:02 am
So a funny thing occurred to me while fiddling with LINQ in Sphere Studio: This (Link.js) is actually more powerful than its namesake. :P  For example there's no easy way in LINQ to iterate over two or more collections as if they were one, you have to concatenate them first.  Whereas in Link, you just do this:

Code: (javascript) [Select]

Link(arr1, arr2, arr3)
    .where(...)
    .etc()


I use the above in the Specs battle engine a lot.  It's handy for iterating over both PCs and enemies in one query (for turn resolution, e.g.), which keeping them neatly in separate arrays for other operations.

There are also a lot of constructs that Link has which are technically possible with LINQ, but are more clunky (requiring the ugly extension method syntax) or just not as obvious.  So, long story short: Awesome work on this. :)
Title: Re: Link.js v0.3.0
Post by: Radnen on October 16, 2015, 03:09:25 am
I'm glad to hear that. I looked at LINQ as inspiration but added things I thought would make it easier to use. :)

I'm still looking into ways to expand this. I mean it's still at 0.3.0 which means that I think it's still in beta. For instance I really want to get started on a backwards traversal mechanism which is basically running the logic backwards. Then I was thinking of branching out to trees (pun intended) and then other data structures.

Thanks for the feedback! You've no doubt been a great help by using it. :)
Title: Re: Link.js v0.3.0
Post by: Fat Cerberus on March 18, 2016, 02:06:23 am
Interesting discovery: Using Link in TypeScript is nearly indistinguishable from LINQ method syntax in C# (capitalization notwithstanding):

Code: (javascript) [Select]

var neoHippos = [
{ name: "maggie", fatness: 812 },
{ name: "Beverly", fatness: 999 },
{ name: "Gail", fatness: 1208 },
{ name: "Burke", fatness: 500 },
{ name: "Machel", fatness: 100 },
];
link(neoHippos)
.where(nh => nh.name != "maggie")
.execute(nh => DebugPrint("neo-Hippo member:", nh.name, nh.fatness))
.each(nh => Print(nh.name + " is really hungry!");


Incidentally, the above code shows a really great use case for .execute() as well. :)
Title: Re: Link.js v0.3.0
Post by: Radnen on March 18, 2016, 11:49:33 am
Nice one. I updated the first page description. It now mentions minisphere and an example for TypeScript. I even removed the legacy code gist that I had embedded.
Title: Re: Link.js v0.3.0
Post by: Fat Cerberus on March 18, 2016, 12:47:17 pm
One area of game development where I've found Link to be extremely useful (and the reason it's included with minisphere) is in implementing things like status conditions, where statuses often have complex clauses determining how they affect various things.  For example, the special Final Stand status used by the final boss of Spectacles: Bruce's Story contains the following Link query:

Code: (javascript) [Select]

link(eventData.action.effects)
.where(it => it.targetHint == 'selected')
.where(it => it.type == 'damage')
.each(effect =>
{
var oldPower = effect.power;
effect.power = Math.round(effect.power / this.fatigue);
if (effect.power != oldPower) {
console.log("Outgoing POW modified by Final Stand to " + effect.power);
console.append("was: " + oldPower);
}
});


Enemy AI benefits greatly from the functionality as well:
Code: (javascript) [Select]

var isPhysical = link(action.effects).filterBy('type', 'damage').pluck('damageType').contains('physical')
                 || link(action.effects).filterBy('type', 'damage').pluck('element').contains('earth');


Or even this masterpiece:
Code: (javascript) [Select]

if (this.tactics === null) {
var targets = link(this.aic.battle.enemiesOf(this.aic.unit)).shuffle();
var combos = link(link(this.combos)
.where(function(combo) { return phase >= combo.phase; }.bind(this))
.random(targets.length))
.sort(function(a, b) { return b.rating - a.rating; });
this.tactics = [];
for (var i = 0; i < targets.length; ++i) {
this.tactics.push({ moves: combos[i].moves, moveIndex: 0, unit: targets[i] });
}
}
this.tactics = link(this.tactics)
.where(function(tactic) { return tactic.unit.isAlive(); })
.where(function(tactic) { return tactic.moveIndex < tactic.moves.length; })
.toArray();
var tactic;
do {
tactic = RNG.sample(this.tactics);
} while (tactic === this.tactics[0] && tactic.moveIndex == tactic.moves.length - 1
&& this.tactics.length > 1);
this.aic.queueSkill(tactic.moves[tactic.moveIndex], tactic.unit.id);
++tactic.moveIndex;
if (this.tactics[0].moveIndex == this.tactics[0].moves.length) {
this.tactics = null;
}


Which basically translates in plain English to "pick a random combo for each party member, and use the finisher for the most powerful one last so that the other ones are decoys".
Title: Re: Link.js v0.3.0
Post by: Fat Cerberus on April 03, 2016, 12:55:21 pm
So my XML parser produces a DOM like in the attached screenshot and I'm wondering what the best way to query it is.  If it's not totally clear from the screenshot, the object model is basically this:

* The DOM has a list of nodes, stored as an array .nodes
* A node may either be a tag or text
* Text nodes are leaves and don't go any further down
* Tag nodes have their own .nodes array with the text and tags contained by them, and .attributes which is a dictionary of the tag's attributes
* And so forth

I can enumerate the top level easily enough with link(dom.nodes),what I'm wondering is how I can drill down through the heirarchy.  Basically if I have the following XML doc:
Code: (xml) [Select]

<neoHippos>
    <pig name="maggie" weight="812 tons">
        <food>Lizzie</food>
        <food>Chiroptera</food>
        <food>Gail</food>
        <food>non-kitty-eating cows</food>
    </pig>
    <cow name="Kittycow">
        <food>Prance</food>
        <food>Shrek</food>
        <food>kitties</food>
    </cow>
</neoHippos>


How could I use Link to enumerate only the <food> elements under <pig>, given the DOM described above?
Title: Re: Link.js v0.3.0
Post by: Radnen on April 03, 2016, 02:47:25 pm
This should work, .recurse will recursively go through sub-arrays if all items are set up the same. I made it to go through filesystem structures a bit easier. If it encounters a node without a 'nodes' it knows to skip it. The where clause knows to check just nodes named 'pig'. The rest is history. :)

Code: (javascript) [Select]

Link(dom).recurse('nodes').where(function(item) { return item.name == 'pig'; }).each(function(item) { console.log(item); });
Title: Re: Link.js v0.3.0
Post by: Fat Cerberus on April 03, 2016, 03:02:24 pm
Does .recurse() unroll the whole structure, or only one level at a time? i.e. Could I do this:

Code: (javascript) [Select]

Link(dom)
    .recurse('nodes')
    .where(x => x.type == 'tag' && x.name == 'neoHippos')
    .recurse('nodes')
    .where(x => x.type == 'tag' && x.name == 'pig')
    .recurse('nodes')
    .where(x => x.type == 'tag' && x.name == 'food')
    .recurse('nodes')
    .where(x => x.type == 'text')
    .each(element =>
{
    console.log("the pig eats: " + element.text);
});
Title: Re: Link.js v0.3.0
Post by: Radnen on April 03, 2016, 03:07:24 pm

Does .recurse() unroll the whole structure, or only one level at a time? i.e. Could I do this:

Code: (javascript) [Select]

Link(dom)
    .recurse('nodes')
    .where(x => x.type == 'tag' && x.name == 'neoHippos')
    .recurse('nodes')
    .where(x => x.type == 'tag' && x.name == 'pig')
    .recurse('nodes')
    .where(x => x.type == 'tag' && x.name == 'food')
    .recurse('nodes')
    .where(x => x.type == 'text')
    .each(element =>
{
    console.log("the pig eats: " + element.text);
});



It unrolls the structure. It checks if the current element has nodes and then enters it and checks if that element has nodes. So, you can't do that. To unroll one level at a time I guess you could use pluck and where:

Code: (javascript) [Select]

Link(dom)
    .pluck('nodes')
    .where(x => x.type == 'tag' && x.name == 'neoHippos')
    .pluck('nodes')
    .where(x => x.type == 'tag' && x.name == 'pig')
    .pluck('nodes')
    .where(x => x.type == 'tag' && x.name == 'food')
    .pluck('nodes')
    .where(x => x.type == 'text')
    .each(element =>
{
    console.log("the pig eats: " + element.text);
});


Pluck just doesn't detect if the 'nodes' property doesn't exist (the assumption being, you ought to know the item to pluck already), but since you filter to only 'tag' types, I think it might be fine.

Edit: but this will act almost no different than .recurse and where, recurse is just neater. Though this pluck-where method could be faster since it doesn't have to throw away as many "junk nodes".
Title: Re: Link.js v0.3.0
Post by: Fat Cerberus on April 03, 2016, 03:14:03 pm
Yeah, tags always have a .nodes which is an array, even if they are self-closing (in which case it's empty).  I'll try it out with pluck and see what happens, thanks for the help. :)

If I use recurse, is there a way to figure out where I came from?  Since it unrolls everything each() only sees the element at the lowest level, with no idea where we came from.  That's why I was looking for a way to selectively drill down.
Title: Re: Link.js v0.3.0
Post by: Radnen on April 03, 2016, 03:26:22 pm

If I use recurse, is there a way to figure out where I came from?  Since it unrolls everything each() only sees the element at the lowest level, with no idea where we came from.  That's why I was looking for a way to selectively drill down.


Unfortunately no, but I do know what you mean. I think that's just the nature of recursive loops in general. It just kind of does it's own thing and you are are the mercy of it's automatic behavior.
Title: Re: Link.js v0.3.0
Post by: Radnen on May 23, 2016, 03:14:04 pm
Feature idea: the cross product!

Code: (javascript) [Select]

// old way:
Link(my_array).each(function(item) {
    var found = Link(other_array).first(function(i) { return i.ID == item; });
    if (found) { found.added = 1; }
});

// new way:
Link(my_array).cross(other_array, function(item1, item2) {
    if (item2.ID == item1) { item2.added = 1; }
});

// experimental non-blocking cross product, creates an interstitial cross-result object.
Link(my_array).crossObject(other_array).each(function(result) {
    if (result.B.ID == result.A) { result.B.added = 1; }
});

// or, using .where:
Link(my_array).crossObject(other_array).where(function(result) { return result.B.ID == result.A; }).each(function(result) { result.B.added = 1; });

// in typescript:
Link(my_array).crossObject(other_array).where(result => result.B.ID == result.A).each(result => result.B.added = 1);


The idea is I want to eliminate the need to have Link contexts unnecessarily inside of other link contexts. Item1 is from the parent Link context, Item2 is from the added link context. BTW, I'm going to modify Link so that every place where it may take an array, could take another link context.

Either I'm going to run the new context's array or I'm going to resolve the new context's chain before running the resulting array. Which do you guys think is appropriate? I'm thinking the resolve approach is better, but you'd have to know that it would resolve it before it is executed. Here is an example:

Code: (javascript) [Select]

var test = Link(array1).where(even);
Link(test).each(print);


In the above example, it will look at the array inputs, realize there is a link context, run it (find all even #'s), then use it's result to print the values (2, 4, 6, 8...). What do you think?

Edit:
Added .where example for the non-blocking code
Title: Re: Link.js v0.3.0
Post by: Fat Cerberus on May 23, 2016, 03:42:23 pm
Link contexts are conceptually arrays, so it makes sense to make them interchangeable, I like it. :)

Not sure I understand the cross-product thing.  I know what a cross product is in math, just not sure how it pertains here.  For the resolving thing, I think it would be nice to maintain the lazy evaluation behavior of possible.  Not sure how difficult that'd be to implement though.
Title: Re: Link.js v0.3.0
Post by: Radnen on May 23, 2016, 04:50:54 pm

Not sure I understand the cross-product thing.  I know what a cross product is in math, just not sure how it pertains here.


Basically it does a 2D search of all value permutations. If you look at the code example you can see how it simplified the code a lot, perhaps even giving it a performance boost.


For the resolving thing, I think it would be nice to maintain the lazy evaluation behavior of possible.  Not sure how difficult that'd be to implement though.


Hmm, yeah, but the final results would need iteration on so it can only be so lazy until you run it. I wonder if I can expose a private API so I can "manually" step through a link execution chain and only execute what I need on-the-fly:

Code: (javascript) [Select]

var array = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10];
var ctxt = Link(array).where(even).yield('toArray');

var v1 = ctxt.step().step().step().step().step().end(); // [0, 2, 4]
var v2 = ctxt.step(5).end(); // [0, 2];
var v3 = ctxt.run(); // [0, 2, 4, 6, 8, 10]


Kind of experimental, but it will ensure that only 'n' steps()'s get executed. That's as lazy as it gets.
Title: Re: Link.js v0.3.0
Post by: Fat Cerberus on July 25, 2016, 01:30:55 am
Good news: Link will officially be part of the standard library in minisphere 4.0. ;D   How you will use it:

Code: (javascript) [Select]

// at top of file (usually):
const link = require('link');

// Link query
tactics = link(allTactics)
    .where(function(it) { return it.target.isAlive(); })
    .where(function(it) { return it.moveIndex < it.moves.length; })
    .toArray();
Title: Re: Link.js v0.4.0
Post by: Radnen on July 27, 2016, 12:45:30 am
I've updated Link to 0.4.0 (https://github.com/Radnen/link.js/releases/tag/v0.4.0)

Changes



The Cross Product:
Code: (javascript) [Select]

var items1 = ["A", "B"];
var items2 = ["C", "D"];
Link(items1).cross(items2).each(function(item, index) {
    Print("Item at: "+ index);
    Print(item); // e.g: ["A", "C"] or ["B", "D"]
});


Index is the number 0..n where n is the number of total items in the final list after the cross product has resolved.
Item is an array whose contents [0..m] is the m times of cross products you do. In the above example, it will contain an array with indices at [0, 1].

You might have noticed in another thread here I had used item.A, item.B, and item.C to denote the items that result from the cross product, for each join. But I deemed the internal method to figure that out too slow. A basic array with the items is sufficient enough. It's now [item1, item2, itemN, ...]
Title: Re: Link.js v0.4.0
Post by: Fat Cerberus on July 27, 2016, 12:54:27 am
It occurs to me that cross() could probably be used to implement matrix multiplication.  Not very practical (there are much more efficient ways to do it), just a curiosity.
Title: Re: Link.js v0.4.0
Post by: Fat Cerberus on November 10, 2016, 02:02:52 pm
Quick suggestion: Rename .each() to .forEach() for consistency with JS array.forEach and also highlighting that it's an endpoint and not lazy like the rest of the chain.  And then .execute could be named .each.  This way it's easy to tell that the two methods do the same thing, the only difference is lazy vs. not lazy.

edit: I found a way to use Link to work directly with objects and query both the keys and values:
Code: (javascript) [Select]

link(Object.keys(tests))
.where(function(key) { return key.substring(0, 4) === 'test'; })
.where(function(key) { return key !== 'test' })
.map(function(key) { return tests[key]; })
.each(function(test)
{
if (typeof test === 'function')
test();
else if (typeof test === 'object' && test !== null)
run(test);
});
Title: Re: Link.js v0.4.0
Post by: Radnen on November 12, 2016, 12:58:57 am
Alright, I added a .forEach alias so it can relate to other libraries. I didn't want to make each become execute or an alias of it because it may break a lot of existing code (I use this library at work professionally, and so do my coworkers, lol).

Your keys method got me thinking and I added a chainable .keys() method which basically enumerates through all of the keys in an array of objects, or you can use pluck to narrow down on to a single object in which to enumerate it's keys. And if you keep a reference to that object, you may decide to retrieve it's values. This method is still, I shall say, experimental.

Example:
Code: (javascript) [Select]

var obj = { a: 1, b: true, c: "hi" };
Link([obj]).keys().forEach(function (key) { console.log(key); }); // "a" "b" "c"
Link([obj]).keys().forEach(function (key) { console.log(obj[key]); }); // 1 true "hi"
Title: Re: Link.js v0.4.0
Post by: Fat Cerberus on November 12, 2016, 01:10:25 am

Alright, I added a .forEach alias so it can relate to other libraries. I didn't want to make each become execute or an alias of it because it may break a lot of existing code (I use this library at work professionally, and so do my coworkers, lol).


That's okay, I might do some tidying up for the version of Link included with minisphere to make things more elegant since Sphere v2 hasn't been finalized yet so I don't have any API compatibility promises in place yet. ;)  But I know what you mean about everyone depending on it, Link is incredibly useful.  Every time I go to write a new miniRT module now the first thing I end up doing is require('link') and I always find some use case for it.  I rarely write for loops in JS anymore. :D

But yes, just seeing something like Link(obj).where(...).each(...) makes me think that the each is actually lazy when it's not.  forEach() makes it clear that we're running the query at that point and iterating on the result.
Title: Re: Link.js v0.4.2
Post by: Fat Cerberus on November 12, 2016, 01:46:35 am
@Radnen: Hm, it looks like the latest changes didn't actually make it into 0.4.2?  The last commit is a README edit from July.
Title: Re: Link.js v0.4.2
Post by: Radnen on November 12, 2016, 03:05:47 am
I think I forgot to push the source with github for windows (ugh!) it keeps updating and I completely spaced out on the 'sync' button. I usually use git solely from command line, and so I just now pushed it via console command line and now everything's good to go!
Title: Re: Link.js v0.4.2
Post by: Fat Cerberus on November 12, 2016, 10:28:15 am
That .keys() method is nice.  A feature suggestion for the future: If user passes a non-array object directly to Link() constructor, apply .keys() automatically, and maybe pass the key and value to control functions (value first for compatibility with array behavior):

Code: (javascript) [Select]
// note: ES6 notation because it's more concise
Link(obj).forEach((v, k) => console.log(`${k} = ${v}`));


That syntax would be very elegant: "for each 'v' with key 'k', print k and v"

edit: Oh, and it also avoids an extra .map() if you need to work with the values instead of the keys.
Title: Re: Link.js v0.4.2
Post by: Radnen on November 12, 2016, 01:04:08 pm
The second argument is already the index, and it can be tricky to assume the array will have objects with keys, because you could be iterating through numbers, or even a string such as Link('hello world!').forEach().

I'd have to add another modifier such as .keyValue() or have keys() itself permanently alter the .each/.forEach parameters...
Title: Re: Link.js v0.4.2
Post by: Fat Cerberus on November 13, 2016, 01:06:31 pm
So I got bored and decided that as a learning experience I should try reimplementing Link from scratch.  The result is this experimental "from" module that natively allows querying objects:
https://github.com/fatcerberus/minisphere/blob/master/assets/system/modules/from.js

Code: (javascript) [Select]

const from = require('from');

// ES6 syntax
var food = { pig: "people", cow: "kitties", ape: "bananas" };
var result = from(food)
    .where((v, k) => k === 'pig')
    .besides((v, k) => console.log(`${k} = ${v}`))  // debug
    .select((v, k) => "812 " + v);

console.log("Which animals survived the eating? " + Object.keys(result));
console.log("What did the pig eat? " + result.pig);


select works like a combination of map and toArray in Link.  If no mapping function is provided it selects the original value (map itself is still available and is chainable).  And besides is like execute: it calls the function and then passes through the original value, ignoring the return value.

No idea how performance compares to Link, though.  There's no manual optimizations, I just implemented things in whatever way seemed most obvious.
Title: Re: Link.js v0.4.2
Post by: Radnen on November 13, 2016, 06:24:54 pm
Ok, that's cool. So you've basically reimplemented link just so you can have value, key pairs in the callback functions, and you seem to make use of more ES6 features.

There's this bit I'm curious about:
Code: (javascript) [Select]

var obj = Reflect.construct(LinkObj, arguments);


This is interesting, because I couldn't find another function that's similar. Using new and apply() doesn't work since Apply will destroy the original constructor. I had the same problem before and simply used arg1, arg2, arg3 manually because there wasn't a decent way of trying to get the arguments to pass to an object instantiation via 'new'.

But I've held off on ES6 for link, and I'm sure my runners and other methods could be simplified a lot by using these new ES6 API's especially the Reflect object.
Title: Re: Link.js v0.4.2
Post by: Fat Cerberus on November 13, 2016, 07:21:36 pm
There's the key/value thing but I also did it as a way to understand how Link actually worked.  I was never able to fully grasp exactly how your chain system worked which made me hesitant to try to make modifications.  Now that I actually programmed something like it I understand how things work a lot better.

Reflect.construct() is indeed nifty.  You can polyfill it for ES5 but it's a bit obtuse.  Basically you do new (constructor.bind.apply(undefined, args)).  The ES6 syntax is much nicer though. :)

One thing I changed over your design was to distinguish chainable actions ("links") from endpoints ("ops").  This allowed me to reuse a lot of code.  It turns out that almost everything can be implemented in terms of map so there was no need to have a bunch of different link types.
Title: Re: Link.js v0.4.2
Post by: Radnen on November 13, 2016, 09:26:56 pm

It turns out that almost everything can be implemented in terms of map so there was no need to have a bunch of different link types.


Yeah, many other systems stop there. Map is everything. Well, technically, reduce, filter and map are the three pinnacles of functional programming since everything you do can map down to one of those three (I even used the word map in that sentence, so they technically reduce to just map). I just add more options to the table since some things make more intuitive sense when everything's not simply 'map'. Because map, and filter is so vague... what exactly does 'unique' do? Well, it's no different than a filter, but it makes sure it produces just the unique results. What does pluck do? Well, it retrieves a certain property, which you could have just done with a basic map.

Reduce is interesting since there is leftReduce and rightReduce (leftReduce is simply 'reduce' and rightReduce is not implemented in Link). It's a combiner.

Code: (javascript) [Select]

Link([1, 2, 3]).reduce((a, b) => a + b).each(console.log); // 6


It looks at 1, and 2 and does the mapping function. Then it takes that result and looks at 3 and does the mapping function again. .each() is not the same thing since it looks at only one item at a time, in effect, it's not "reducing" the size of the set, like how the above operator did. Hashing algorithms and checksums are often implemented this way.

Edit:
rightReduce puts the result i n 'b' while leftReduce puts the result in 'a'. Right reduce is intended to work backwards in a list, which is not easy to achieve in link since it only goes forwards.

Link.js idea: make an option for it to go backwards... Could be interesting...
Title: Re: Link.js v0.4.2
Post by: Fat Cerberus on November 14, 2016, 02:49:33 pm
Yeah, the extra conveniences are a big part of what keeps me using link.  Especially since ES5 doesn't have arrow functions, it's much easier to figure out what pluck('name') does vs. an equivalent map operation.  It's a pain to do functional programming in pre-ES6 JavaScript because lambdas are so damn verbose.

I'm experimenting with simplifying things some more for my experimental "from" script.  One thing I notice that link does is to call the points recursively.  While this is in line with the chain metaphor, it ends up duplicating effort because every single point (except endpoints) needs to have a call to this.next.exec().  My idea was to instead have points return true to continue, and false to drop the current value, filtering it out.

Endpoints would work similarly - return true to continue with the next value, or false to short-circuit.  I think this is overall a better separation of concerns.

One thing I'm having difficulty understanding is how link adds items to a query midstream.  For example link(a1) starts out with all of a1 but then you can tack on .cross(a2) and get a bunch of permutations.  Not sure how that works.  Also what does coalesce do?  It says it merges modified results back into the array, but I'm not sure what that means in practical terms.  Link already has update()...
Title: Re: Link.js v0.4.2
Post by: Fat Cerberus on November 15, 2016, 02:38:23 am
Just going to leave this here ;)

Code: (javascript) [Select]
from.Array(threads)
.where(function(t) { return t.id == threadID })
.besides(function(t) { t.isValid = false; })
.remove();

// you could also do `from.Object(obj)` to query keys, or just plain `from()` to determine
// the enumeration mode from the type of the argument.


That reads back very nicely I think: "From Array 'threads', where an item's id matches the thread id, besides setting 'isValid' to false, remove it."
Title: Re: Link.js v0.4.2
Post by: Radnen on November 15, 2016, 09:15:38 pm

I'm experimenting with simplifying things some more for my experimental "from" script.  One thing I notice that link does is to call the points recursively.  While this is in line with the chain metaphor, it ends up duplicating effort because every single point (except endpoints) needs to have a call to this.next.exec().  My idea was to instead have points return true to continue, and false to drop the current value, filtering it out.

Endpoints would work similarly - return true to continue with the next value, or false to short-circuit.  I think this is overall a better separation of concerns.


I actually had this exact implementation in Link, but for some reason I got rid of it... Not sure why though... I think the performance was better to not return a value?


Also what does coalesce do?  It says it merges modified results back into the array, but I'm not sure what that means in practical terms.  Link already has update()...


It is similar, yes. But with update you can specify a value. Coalesce uses the current computed value in the Link list and sticks it back into the array at the index value, but it's still experimental because some operations might change the index value.
Title: Re: Link.js v0.4.2
Post by: Fat Cerberus on November 16, 2016, 12:12:31 am
Okay, so in other words:

Code: (javascript) [Select]
var nums = Link.range(5);
Link(nums)
    .map(function(n) { return n * n; })
    .coalesce();

// 'nums' is now -> [ 1, 4, 9, 16, 25 ]


That's pretty convenient, basically transforming the entire array in one step.  Or you could add where clauses to only remap certain items... hm... I'll have to try to find a use for this now :)
Title: Re: Link.js v0.4.2
Post by: Fat Cerberus on November 16, 2016, 10:48:15 pm

Link.js idea: make an option for it to go backwards... Could be interesting...


Just saw this bit.  That would indeed be interesting to see.  Among other things, it would allow last to be able to short-circuit the same way first can.  When going forwards you always have to run the entire sequence to find the last match(es), if you start from the end you can just stop after you have X items and then reverse them.
Title: Re: Link.js v0.4.2
Post by: Fat Cerberus on November 18, 2016, 11:57:03 am
Hm, it occurs to me that one can take advantage of lazy evaluation to implement random chances for battle engines:

Code: (javascript) [Select]

const link   = require('link');
const random = require('random');

// insta-kill attack
link(battlers)
    .where(function(b) { return b.isEnemyOf(attacker) })
    .where(function(b) { return b.level % 5 == 0; })    // level is multiple of 5
    .where(function() { return random.chance(0.35); })  // 35% hit rate
    .each(function(b) { b.die(); });
Title: Re: Link.js v0.4.2
Post by: Fat Cerberus on November 21, 2016, 12:40:22 pm
Feature request: Partial application, or in other words reusing a partial query with different operators.  Link can already kind of do this, but only with endpoints.  In C#, you can do stuff like this:

Code: (csharp) [Select]

// get all items for the first party member:
itemQuery = items.Where(v => v.owner == partyChars[0]);

// get items to show on the current page
itemsOnPage = itemQuery
    .Skip(pageSize * pageIndex)
    .Take(10)  // 10 items per page
    .ToArray();


The above doesn't work in Link because adding points just keeps adding to the same query.  LINQ (and soon, Sphere v2 from()) returns a new enumerable for each point, which allows this to work.
Title: Re: Link.js v0.4.2
Post by: Fat Cerberus on February 04, 2017, 03:31:26 am
Let me just say that Link and from() are so much more fun to use now that minisphere supports arrow functions. :D
Title: Re: Link.js v0.4.2
Post by: Fat Cerberus on February 28, 2017, 10:57:42 am
I found the first actual use case for from() over objects:
https://github.com/fatcerberus/spectacles-i/blob/master/src/party.js#L47-L50

Party management. :D