Skip to main content ↑ ↓
Topic: Re: C#/C plugins for Sphere (Read 167420 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

Re: Re: C#/C plugins for Sphere

Reply #1 –
Quote from: Flying Jester – on

That would change if we figured out how to make a C# and C-linkaged fat-binary dynamic library.

That would be unbelievably awesome, but I'm somewhat doubtful if that's possible. If that was done, combined with a V8+OpenGL TurboSphere, would push Sphere firmly into the realm of next-generation 2D game engines. Heck, with something like that, it would probably not be to far to call Sphere the most advanced open source game engine.

So basically it would work something like: Someone writes a map engine, like Majestic, in C, and both the editor and the engine could use it? The editor could edit Majestic maps and the engine could run them? Wow, that would be cool. (I'm just using Majestic as a random example, but wouldn't it be cool if engine and editor plugins could be written in JS as well as C?)

Re: Re: C#/C plugins for Sphere

Reply #2 –
With reflection it is not impossible to create bindings between the two. But the bindings must be made, what we want here is a seamless transition, right? C# and C would have to compile to an IL before that could happen. Mono + CIL would then have to be the answer here.
If you use code to help you code you can use less code to code. Also, I have approximate knowledge of many things.

Sphere-sfml here
Sphere Studio editor here

Re: Re: C#/C plugins for Sphere

Reply #3 –
I've been reading about it. The usual thing to do is to use C++/CLI. Which sounds fine on Windows. Not so much anywhere else.

What I'm really imagining is a shared library that has some symbols that are visible and use the C# conventions, and some symbols that have C linkage.

Depending on what a C# DLL needs to be, though, it still might be possible. The linkage and symbols needed for TurboSphere are very simple (theoretically, you only need to return a list of addresses, some ints, and some C strings, and you could get the addresses from, say, a statically linked library that is linked to the plugin), and there should be a way to have some symbols in a DLL or SO be C# and some be C? That has to be possible. I know that theoretically, it is. All the SOs and DLLs for TurboSphere have mixed C and C++ linkage, and in the past I've made DLLs that had mixed Pascal and C linkage, although in both those cases the compiler supports making the binary file that way.

It's a slightly unnecessary idea, though, you could just write the a library and have two separate plugins for the engine and editor that both call the library. But it would just be so incredibly cool and streamlined to have a single DLL that works as both, and is being called from both C/C++ and C#.

Radnen, when you make a DLL with C#, what kind of intermediate object files you get? Is it a similar process to the old Windows compilers before .NET, and the C/C++ compilers?
 

Re: Re: C#/C plugins for Sphere

Reply #4 –
Quote from: Flying Jester – on

I've been reading about it. The usual thing to do is to use C++/CLI. Which sounds fine on Windows. Not so much anywhere else.


That why: Mono.

Quote from: Flying Jester – on

Radnen, when you make a DLL with C#, what kind of intermediate object files you get? Is it a similar process to the old Windows compilers before .NET, and the C/C++ compilers?


I ... don't know! I don't know how to show the individual object files when building a C# program. I'm sure there is a way though.
If you use code to help you code you can use less code to code. Also, I have approximate knowledge of many things.

Sphere-sfml here
Sphere Studio editor here

Re: Re: C#/C plugins for Sphere

Reply #5 –
They don't get left over in an intermediate directory (that way you don't need to recompile those parts if the code hasn't changed for them when you compile again)?

I think (I don't use .NET much) that I'm thinking of the object files the IL would ouput. I'm sort of assuming the whole compilation process is similar to how GCC/MSVC works, just with a C# to IL compiler first, which could be very wrong. Just C# is compiled to IL, which is compiled to object files, which are linked to create a binary. Which could be not how it works at all.

I'm hoping that the pre-linking object files may be the same format across MS development tools. In which case, I'm hoping to do some linker magic and just have it all work out. Otherwise...I'm not sure what, otherwise.

Re: Re: C#/C plugins for Sphere

Reply #6 –
Quote from: Flying Jester – on

It's a slightly unnecessary idea, though, you could just write the a library and have two separate plugins for the engine and editor that both call the library. But it would just be so incredibly cool and streamlined to have a single DLL that works as both, and is being called from both C/C++ and C#.

There's a big cool factor and a very big convenience factor to having just one DLL. Imagine: you download particleSystem.dll (possibly even through the editor) and you get a particle editor in Sphere Studio and TurboSphere suddenly grows particle capabilities.

That's got me pretty excited. Maybe I'll prototype some possible features for plugins in 1.7. Given the age of Sphere's codebase and the somewhat poor documentation on MDN combined with extensive API changes, SpiderMonkey upgrades will be very hard to pull off, so I think I'll resort to the current Sphere implementation being a testbed for new ideas (which it sort of already is, with the particle system). If you guys don't mind, I think that's the direction I'll take with Sphere 1.7: new features, instead of updated libraries like I originally planned.

Re: Re: C#/C plugins for Sphere

Reply #7 –
That sounds fine to me!

Yeah, the unified DLL/SO thing is really an aesthetic, and partially a convenience. But if it worked, it would also really impress folks who knew what they were looking at--me included! Plus, I'm, not sure anyone else has done just exactly what we're talking about, and this the perfect use for it.

About how I am hoping to do it, I'm hoping that object files generated by MSVC are the same format as the ones made by the MS C# .NET compiler at some point along the way, and one or the other linker (if they are different) could be used to link both the TS plugin's object files and the editor's plugin's object files together and make a single binary. I highly doubt that would work, but it is a good thing to try first I think. The same might work (and I actually suspect will work better) using LLVM or GCC on Linux and Mac.

Re: Re: C#/C plugins for Sphere

Reply #8 –
Quote from: Flying Jester – on

Yeah, the unified DLL/SO thing is really an aesthetic, and partially a convenience. But if it worked, it would also really impress folks who knew what they were looking at--me included! Plus, I'm, not sure anyone else has done just exactly what we're talking about, and this the perfect use for it.

It just sounds awesome to say, "Yeah, just download particleSystem.dll and put it in Sphere/plugins/" as opposed to downloading several plugin DLLs and putting them in various folders (although if there were a central repository the editor might be able to automate that). But it would still be very nice for distributing plugins on the forums: you'd just have to attach one file for people to download and put in one location.

Quote

About how I am hoping to do it, I'm hoping that object files generated by MSVC are the same format as the ones made by the MS C# .NET compiler at some point along the way, and one or the other linker (if they are different) could be used to link both the TS plugin's object files and the editor's plugin's object files together and make a single binary. I highly doubt that would work, but it is a good thing to try first I think. The same might work (and I actually suspect will work better) using LLVM or GCC on Linux and Mac.

Unfortunately, C# doesn't actually generate object files; it just compiles everything at once, every time (this is a side effect of not having header files). The C# compiler is fast enough that this is barely noticeable. :(
I saw this in an article by Eric Lippert recently, who seemed to think it was a good idea (avoids the inconveniences of header files, like writing every function definition twice).

EDIT: Found it: http://blogs.msdn.com/b/ericlippert/archive/2010/02/04/how-many-passes.aspx

Re: Re: C#/C plugins for Sphere

Reply #9 –
I figured this was going off the topic of my Website page and so split it here so it can be found and read a bit more easily. :)

I already knew C# had a multi-pass compiler, so that it didn't require header files (it technically does, it's just that they are the same file as the code file). C++ could easily have a compiler just like how C# works. Anyways, C++ also goes to CIL. I wonder, what happens if TurboSphere was built with Mono's C++ compiler? Does mono even convert C++ into MSIL ("CIL") or the Mono version of it? If so, could the resulting library just be used as if they were any other .NET/Mono DLL?
If you use code to help you code you can use less code to code. Also, I have approximate knowledge of many things.

Sphere-sfml here
Sphere Studio editor here

Re: C#/C plugins for Sphere

Reply #10 –
TurboSphere has accidentally turned out to be written in pretty portable C++ (it has moved from MSVC '08 to 10 to GCC C++ to C++11 with only new warnings each time). It wouldn't surprise me if it also compiled as C++/CLI too.

I've sort of been hoping to add on the C-linkage plugin interface from TurboSphere on the end of a .NET DLL, since C-linkage is very simple, and the plugin interface I specified is very simple since it means that both the plugin side and the engine side talk to V8 directly.

I'll have a closer look at Mono (I don't really know that much about it) and see what it would take, if it is possible, to compile (or..something) TurboSphere to use it (?).

Re: Re: C#/C plugins for Sphere

Reply #11 –
All I know is that C++, VB, and C# compile to CIL. Mono is a reimplementation of C#, so it must also compile to an IL (one that is cross platform, that is). In that case, I wonder if MonoDevelop compiles C++ and VB too to their IL, or if it is *just* C#?

Hmm, well this is a good start: http://www.mono-project.com/CPlusPlus, but it has some limitations.
If you use code to help you code you can use less code to code. Also, I have approximate knowledge of many things.

Sphere-sfml here
Sphere Studio editor here

Re: C#/C plugins for Sphere

Reply #12 –
I know that the way I compile TurboSphere with MSVC, it does not use CIL. I think it doesn't anyway. It outputs object files, which the linker links to make a binary. I could be wrong, if the object files are CIL. In which case, the Visual Studio linker should, somehow, be able to take both the CIL object files from a TurboSphere plugin, and the CIL output for a Sphere Studio plugin, and make a mixed library that contains the code for both. But again, I've never heard (and I've looked) of such a thing being done. In fact, I think (and am more sure that) such a thing is possible for any two languages supported by the GCC, since the object code is output by each language front end to be processed by the single backend, but I can't seem to find anything about that, either.

I think I'll get some MS C#, and see how it works. If this even just worked on Windows, that would be amazing. Radnen, is the code for a Sphere Studio plugin available online? Even an unfinished one. I would like to experiment with this.

Making a single routine handle, say, opening a filetype, when the editor and the engine ask, is another story. I'm not worried about that yet. Even if that isn't possible, the same end result can be achieved.

Another problem is that a TurboSphere plugin needs to also have C++ code internally. The plugin interface, while only given C-linkage, gives the engine addresses to C++ functions. A plugin needs the STL library to talk to V8, and the plugin directly calls V8 functions and V8 quite extensively uses templates. The reason I could implement the plugin system so fast was that I unloaded all the heavy lifting onto the plugins, the engine does very little now. It mainly starts up a V8 context, compiles scripts (which could easily be a plugin's responsibility), and intializes plugins. To change that, I would have pretty much abstract away the interface to JS. Which is an immense undertaking, and begins to tempt that black hole of total extensibility.

EDIT:

Apparently, we aren't the first to seriously wonder. http://blogs.msdn.com/b/texblog/archive/2007/04/05/linking-native-c-into-c-applications.aspx. And apparently, the MSVC linker is the answer? I'm going to give it a shot.

If anyone else wants to cast their lot in, I created the plugin getkeystring just for testbed scenarios (like this, but I actually expected the first use would be for making plugins entirely in other languages, not for experimenting with kind-of-fat binaries).

Re: C#/C plugins for Sphere

Reply #13 –
Well...

This might not be so hard after all.

Check this DLL out. It works as a TS plugin, and I think it implements IPlugin for your editor as well.

To try this at home, from the Visual Studio 2010 command prompt:

Code: [Select]

C:\spherestudio\Sphere.Plugins>csc /target:module
 /r:"C:\GitHub\spherestudio\Sphere Editor\Libs\dockpanel
suite\WeifenLuo.WinFormsUI.Docking.dll"  IPlugin.cs IPluginHost.cs Properties\As
semblyInfo.cs


That makes IPlugin.netmodule.

Code: [Select]

C:\turbospheresrc\plugins\getkeystring>cl /c /
MD getkeystring.cpp


Which makes getkeystring.obj. Put them in the same directory.

Code: [Select]

C:\turbospheresrc\plugins\getkeystring>link /L
TCG /DLL /ASSEMBLYMODULE:IPlugin.netmodule "../../lib/v8.lib" /OUT
:fatplugin.dll getkeystring.obj IPlugin.netmodule


Which makes fatplugin.dll. I have no idea how well it works as a Sphere Studio plugin, but it is a completely working TurboSphere plugin that replaces getkeystring, and is 4kb (about the size of the IPlugin.netmodule) larger than the normal getkeystring.dll.

It would be another matter to have it call common routines from both sides (if the C# side is really working correctly at all). But I for one think this is better already than having two separate plugin files for a single purpose.

EDIT:

Well...I see now what that actually did. Clearly not what is needed to make the C# code I wanted to be put into the dll.

I'm going to try to figure out how to get the command line output from Visual C#.

Re: C#/C plugins for Sphere

Reply #14 –
Quote from: Flying Jester – on

The reason I could implement the plugin system so fast was that I unloaded all the heavy lifting onto the plugins, the engine does very little now. It mainly starts up a V8 context, compiles scripts (which could easily be a plugin's responsibility), and intializes plugins. To change that, I would have pretty much abstract away the interface to JS. Which is an immense undertaking, and begins to tempt that black hole of total extensibility.

With a little C++ template magic, it's not hard at all to bind C++ functions to JS without the C++ knowing much. See the way 0 A.D. (an open-source RTS, which I'm a fairly active contributor to) does it, particularly ScriptFunctions.cpp.

Oh, and that link is very interesting!

Quote from: Flying Jester – on

It would be another matter to have it call common routines from both sides (if the C# side is really working correctly at all). But I for one think this is better already than having two separate plugin files for a single purpose.

That actually wouldn't be too hard. The C# could just make P/invoke calls to the C++ side of it's DLL, so all the functionality would be in C++ and the GUI stuff would be in C#.