hIDE

Greetings,

As you may know, I’m working on a new plugin system for HIDE using the ghc-api. This new system allows the programmer to export arbitrary closures as top-level functions and thus avoids the problem of global mutable variables (for example: newIORef >>= \ref -> export (”myfunc”,myfunc ref)).
The observant reader might object to this and say that the closure environment in the linker can’t contain pointers to arbitrary closures since the closures are moved around by the GC. The reader would be right and, consequently, this approach only works when the plugins are interpreted (in which case an extra closure environment is kept for bindings bound at the REPL).
To get around this, I added a list of root pointers to the linker. The GC can then update the pointers whenever the closures are moved. This simple solution took me days to figure out. I should really write a paper about this.

After much hard work, I’ve finally brought my plug-in ideas into reality.

The first plug-in:

module Test where
	
import Hide.Plugin.Base
import Data.IORef
	
initFunc :: IO ()
initFunc = putStrLn \"First plugin init.\"
	
exporter =
    do ref < - newIORef 0
       let symbols = (\"testSucc\", modifyIORef ref succ) :+:
                     (\"testPred\", modifyIORef ref pred) :+:
                     (\"testShow\", readIORef ref >>= print) :+:
                     HNil
       export symbols

The second plug-in:

module Test2 where
	
initFunc :: IO ()
initFunc = do testSucc
              testSucc
              testShow
              testPred
              testShow

Result of loading Test2:

First plugin init.
2
1
Ending hIDE session

I wasted a whole day trying to figure out why “initFunc” didn’t match “initFunc”. It turned out that reverting all CAFs after loading a plugin also reverted the CAFs in GHC, and consequently, reset the global string table.
I’m not sure what the effects are of not reverting all CAFs after a load but time will tell.

Greetings,

The plug-in architecture of the first incarnation of hIDE was pretty much a disaster. It was unsafe and very difficult to wield, and I honestly couldn’t figure out how to fix it. A change of scenery was needed so I started working on debugging with GHCi (see this mail).
By doing so I learned how to bind new closures dynamically which can be immensely useful. It makes it possible to not only jump from dynamic to static code but also to define top-level curried functions. Here’s how I imagine the new plug-in architecture:

Plug-in Project-registry:

Meta-file:
> depends: {- nothing -}
> provides: project-registry
Source-file:
> module ProjectRegistry ( exportedSymbols ) where
> import HidePluginBase
> -- The type signature will usually be inferred.
> -- exportedSymbols :: IO (HideExport
> --                        (type of registerProject
> --                        ,type of unregisterProject))
> exportedSymbols =
>   do projectRef < - newRef [] -- Probably IORef or MVar.
>      return $
>        Export (registerProject projectRef
>               ,unregisterProject projectRef)
>
> registerProject projectRef project = {- register code -}
> unregisterProject projectRef project = {- unregister code -}

Plug-in Haskell-project:

Meta-file
> depends: project-registry
> provides: {- nothing -}
Source-file:
> module HaskellProject ( initCode ) where
> import HidePluginBase
> initCode :: IO ()
> initCode =
>   do registerProject myHaskellProjectCode
>      return ()

HIDE will make sure that plug-ins are loaded and initialized in the correct order, and that the exported symbols from one plug-in are available in the plug-ins that depend on it. Also, note that plug-ins depends on and provides interfaces. That means that plug-ins can be composed freely (something that wasn’t previously possible because plug-ins depended directly on each other).

However, there’re still a few issues to be resolved.

• This kind of plug-ins can only be compiled to byte-code, and, consequently, may impose a performance problem.
• Since we’re compiling to byte-code, Cabal isn’t an option.
• And lastly, since we can’t use Cabal, plug-ins may only contain Haskell code (no hsc2hs, no c2hs, no cbits).

Luckily, we can work around these problems by moving the core of a plug-in to a Haskell library while keeping the controlling code. Of course, the core will then be static and should be kept as small as possible.

After months of inactivity, the hIDE project is finally moving again.

I’ve learned a lot from hacking the first core of hIDE. Writing a dynamic application of that scale was a completely new experience. However, it’s my opinion that the relatively simple linker system of hs-plugins isn’t quite suited for the task; It’s too massive for the initial boot linker (the small static core that links and executes the dynamic system), and it doesn’t provide enough safety and control for high-level plugin management.
Consequently, I’ve decided to use the GHC-api for the high-level plugin management. The boot linker could theoretically use the GHC-api, too, but that would result in a rather large binary. Instead, it uses its own boot linker (a stripped down version of the GHC linker) to link the dynamic main and transfer control to it.
Building a dynamic application from the ground-up with hs-plugins, necessitates passing the “linker” function to the dynamic system, to allow loading additional plugins. That’s because the global variables in the static core that keep track of dependencies aren’t available from the dynamic system: Loading hs-plugins from the dynamic system will simply initiate a new set of global variables. Moreover, loading the same package twice is a waste of space.
However, it’s only feasible to pass around the linker-API if it’s very simple. Instead, I choose to pass the linker state (list of loaded packages) from the static core to the dynamic system[1]. This is made feasible by the combination of a minimal boot linker and a comprehensive plugin manager.

[1] The GHC-api didn’t support this so a tiny bit of hacking had to be done.

Through the dark ages many a programmer has longed for the ultimate
tool. In response to this most unnerving craving, of which we
ourselves have had maybe more than our fair share, the dynamic trio of
#Haskellaniacs (dons, dcoutts and Lemmih) hereby announce, to the
relief of the community, that a fetus has been conceived:

hIDE - the Haskell Integrated Development Environment.

So far the unborn integrates source code recognition and a chameleon
editor, presenting these in a snappy gtk2 environment. Although no
seer has yet predicted the date of birth of our hIDEous creature, we
hope that the mere knowledge of its existence will spread peace of
mind throughout the community as oil on troubled waters. More news
will be dispersed as it arises.

Letting a plugin kill the parser thread made everything much more stable. Using ‘Control.Concurrent.killThread’ doesn’t always work since the exception is catchable and the threads doesn’t always check what they’ve catched. Eg. ’someIOAction `catch` \e -> return Failed’ would make the thread think something went wrong in ’someIOAction’ instead of shutting down.
Apparently, GHC has a global table of instances which has been the cause of the confusing errors I’ve experienced. To overcome this we make sure that no two instances of the parser is executed simultaneously.
We still got a slight problem with snappiness since we can only use Haskell threads (gtk(2hs) issue). But I’ll leave that to Duncan.

Many people have reported problems while trying to compile ghc-api. While they use a variety of different OSes, they all have one thing in common: GHC fails with ‘unknown expection’.
Fixing the problem may prove to be difficult since I’ve been unable to reproduce this error myself.

Welcome to WordPress. This is your first post. Edit or delete it, then start blogging!