Skip to main content ↑ ↓
Topic: Sphere 2.0 API: SphereFS (Read 266507 times) previous topic - next topic
0 Members and 4 Guests are viewing this topic.

Re: Sphere 2.0 API: SphereFS

Reply #15 –
I've also decided on the following requirement:

* A SphereFS compliant engine must report any path accessible through a SphereFS designator (~sys/ etc.) to be reported as such and never as an absolute path, even if the engine itself supports absolute paths.  This might seem onerous, but it greatly simplifies, e.g. debuggers by canonizing the relative path.

As a practical example of how this is useful: One of the things the Cell compiler will do when compiling a game is generate a source map which maps files in the compiled game to their locations in the original source folder.  The paths in the source map are relative, so if the engine always reports relative paths for its own scripts and other assets, the debugger can always locate the corresponding file in the project with little difficulty.

Re: Sphere 2.0 API: SphereFS

Reply #16 –
To me, there are only two examples of when absolute paths are useful to the engine:

  • Opening a game

  • Saving or opening a resource that the user specifies



The first is obvious, and necessary for most platforms (maybe not necessary for Windows, I am certain not for Haiku, or for XFCE on Unix).
The second is the only reason I really think absolute paths are important to script. This lets you, for instance, save a screenshot to a directory that the user specifies, or load an image to use for a skin from anywhere on the computer. In both those cases, the platform differences between paths also don't matter, since for the most part it will be up the libc on the platform (or the Mon runtime/CLR, etc), and it will know how to handle the platform's paths.

Re: Sphere 2.0 API: SphereFS

Reply #17 –
Yes, another thing I didn't mention above is that SphereFS standardizes on forward slashes as a separator.  / works on all major platforms, so it seemed the best choice.

Basically what my intent with the SphereFS standard is, to make so that a relative path is never ambiguous--"sounds/munch.wav" always refers to the same file regardless of which API is accepting the filename.  This has a few advantages:

* More flexibility for organizing assets without having to resort to ../ trickery
* No extra path processing needed to use a path constructed for one type of API with an unrelated function--opening sound files as rawfiles instead, for instance.
* As mentioned earlier, simplifies debugging by always using the relative path to refer to assets.  This is particularly important when the game being debugged is SPK-packaged.

It also makes package support a lot easier to implement if the relative path is canonical, for reasons I'd imagine are self-evident.

Absolute paths are unambiguous by definition, so supporting them alongside SphereFS is no problem.

Anyway, now that I've got this idea somewhat solidified I wonder if I should submit it as a pull request on the Pegasus repo.

Re: Sphere 2.0 API: SphereFS

Reply #18 –
I have one issue.

Modules.

If I do "require('./file')", how does that work? Do we have two different systems, one being SphereFS for any file operation, and another syntax for require/resolve?

I think we should rethink this SphereFS to work with modules, for Pegasus. In Pegasus you really want relative paths for modules.

I do like having access to the  games user directory: a requirement.
And indeed, absolute paths are neccesary for game loading/resource loading by user.  Prefixing such is dangerous.

Also, I would not make this an extension but core. So you can always depend on it. It is annoying to do:

Code: [Select]

if (engine supports spherefs)
    use path '~usr/save1.json'
else
    use path 'C:.....'


When it is not neccessary, if we just enforce this kind of system.

Re: Sphere 2.0 API: SphereFS

Reply #19 –
SphereFS itself is not intended to be an extension, the only thing I intended to be optional when I designed this is absolute paths since the default case is a sandboxed FS.  The prefixes like ~usr/ would always be available.

As for modules, relative IDs are resolved as described in CommonJS: relative to the calling module.  Logically the module namespace is separate from the file system because any other setup risks breaking CommonJS spec.

I know you're using Node.js as an exemplar when thinking of modules.  Node conflates the CommonJS namespace with the file system and makes for some really confusing semantics which deviate from CommonJS in a lot of cases.  I'll discuss that bit more on IRC.

Re: Sphere 2.0 API: SphereFS

Reply #20 –
kay, I think you need to explain indeed.

But I do want to be able to:

require("random") to get the system random module.
fopen("~usr/game1.json") for savefiles
new Image("~??/images/loadingImage.png") for loading images
require("./teleporters"); to load my own teleporters.js file (in same dir as current module)
require("teleporters"); to load the custom teleporter package.

Speaking of packages, are we using CommonJS-ish packages?

I am on IRC today

Re: Sphere 2.0 API: SphereFS

Reply #21 –
All of those should work as you describe in the final version of minisphere 4.0/Sphere 2.0.  Note that the prefix is not needed for files which are part of the game--that is, the relative path is canonical so you can specify that image as "images/loadingImage.png" and that would be completely unambiguous.  The ~sgm/ prefix could probably be removed altogether, the main reason I added it was for backward compatibility with the Sphere 1.x APIs which are relative to a subdirectory by default.  Pegasus is not backwards compatible so it's not needed there.

Re: Sphere 2.0 API: SphereFS

Reply #22 –
In my Pegasus fork I simplified the SphereFS prefixes.  For Pegasus we will have:

@/ - root of sandbox
~/ - for user data, etc. (like UNIX)
#/ - engine assets

I would love to implement this in the main branch also but care must be taken not to break legacy compatibility.  ~/ means "root of sandbox" in Sphere 1.x. :-\  LoadImage() et al. will probably need a custom path handler to resolve the conflict.
 

Re: Sphere 2.0 API: SphereFS

Reply #23 –
Not sure I like this change. The previous setup (which i implemented 15min ago), is much more clear. Also, ~/ is confusing on *nix systems because it actually means the User folder, not the app-data folder.
Now if a path ~/ is passed for being meant as /Users/name/ it cant be picked up as an error.

When using ~usr/ and ~sys/, it is easy to remember which is which. With @ and # it is not

Re: Sphere 2.0 API: SphereFS

Reply #24 –
~usr/ doesn't actually refer to AppData but rather the user's home directory (or a sub directory thereof, for sandboxing reasons).  I don't particularly like the idea of hiding save data from the user.  And I thought #/ was pretty clear too, since it's often used as shorthand in shells for "you are root" so it makes sense that it refers to the engine itself.

In any case "~usr" is ambiguous, since on a UNIX platform /usr/ isn't actually a user folder but in reality is a system directory.  "UNIX system resources" or something like that, and non-root can't write to it.

Re: Sphere 2.0 API: SphereFS

Reply #25 –
You cant do ~usr/ as just the user home dir... I can't assume writability of ANY place for my app other than /User/<user>/Library/Application Support/NameOfApp/. Neither should you. Same for windows with LocalData.

You said it is a Sandbox, then it should be a true sandbox. No easy access to user files. And no assumptions. That is what I implemented and that is what I really liked about it.

Then lets make it ~pref/ or ~user/ instead of ~usr/ if you think it will be confused with /usr.

I would like to hear some opinions of others as well.

Re: Sphere 2.0 API: SphereFS

Reply #26 –
How is the user's homedir (specifically, their documents folder) not guaranteed to be writable?  That wouldn't be very useful for the logged-in user.  Anyway I'm not a fan of games that hide their save data from me by putting them in Application Data.

And it IS a sandbox.  In minisphere 3.0 ~usr/ refers to <user-documents>/Sphere 2.0/SaveData and on Linux it's mapped to /home/username/Documents/Sphere 2.0/SaveData.  This is documented.  Since it is impossible to do ~usr/../ there is no risk of the game subverting the sandbox.

Long story short, the semantics haven't changed.  Only the prefixes have.

Re: Sphere 2.0 API: SphereFS

Reply #28 –
Fair enough.  Either way, I didn't change the meaning, only shortened the prefixes.  That's all.  Sorry if there was any confusion. :)

Re: Sphere 2.0 API: SphereFS

Reply #29 –
Quote from: Lord English – on
on Linux it's mapped to /home/username/Documents/Sphere 2.0/SaveData


More common would be `/home/username/.share/local/Sphere2/SaveData`, that's how most programs store data like that in Linux. And it's different in real Unix descendants...but Allegro doesn't work there anyway.

Linux/Unix folk will also be unhappy if you start putting spaces in their filenames.