From duncan.coutts at worc.ox.ac.uk Mon Feb 12 18:05:38 2007 From: duncan.coutts at worc.ox.ac.uk (Duncan Coutts) Date: Mon Feb 12 17:59:55 2007 Subject: [C2hs] patch applied (c2hs): Use standard Control.Exception.assert Message-ID: <20070212230538.GA15234@cvs.haskell.org> Thu Feb 8 14:33:56 PST 2007 Duncan Coutts * Use standard Control.Exception.assert In preparation to remove our own version. M ./base/syms/Attributes.hs -12 +5 M ./c2hs/c/CTrav.hs -5 +3 From duncan.coutts at worc.ox.ac.uk Mon Feb 12 18:05:39 2007 From: duncan.coutts at worc.ox.ac.uk (Duncan Coutts) Date: Mon Feb 12 17:59:56 2007 Subject: [C2hs] patch applied (c2hs): Eliminate use of PrettyPrintMode and co Message-ID: <20070212230539.GA15254@cvs.haskell.org> Thu Feb 8 14:39:53 PST 2007 Duncan Coutts * Eliminate use of PrettyPrintMode and co M ./c2hs/c/CPretty.hs -1 From duncan.coutts at worc.ox.ac.uk Mon Feb 12 18:05:40 2007 From: duncan.coutts at worc.ox.ac.uk (Duncan Coutts) Date: Mon Feb 12 17:59:56 2007 Subject: [C2hs] patch applied (c2hs): Eliminate use of errorCodeFatal Message-ID: <20070212230540.GA15268@cvs.haskell.org> Thu Feb 8 14:42:37 PST 2007 Duncan Coutts * Eliminate use of errorCodeFatal M ./base/state/State.hs -2 +2 M ./base/state/StateBase.hs -1 +1 From duncan.coutts at worc.ox.ac.uk Mon Feb 12 18:05:42 2007 From: duncan.coutts at worc.ox.ac.uk (Duncan Coutts) Date: Mon Feb 12 17:59:58 2007 Subject: [C2hs] patch applied (c2hs): Switch from FiniteMaps to standard Data.Map Message-ID: <20070212230542.GA15282@cvs.haskell.org> Thu Feb 8 14:45:01 PST 2007 Duncan Coutts * Switch from FiniteMaps to standard Data.Map and drop bundled FiniteMaps module R ./base/general/FiniteMaps.hs M ./base/syms/Attributes.hs -7 +8 M ./base/syms/NameSpaces.hs -8 +8 M ./c2hs/gen/GBMonad.hs -18 +18 From duncan.coutts at worc.ox.ac.uk Mon Feb 12 18:05:43 2007 From: duncan.coutts at worc.ox.ac.uk (Duncan Coutts) Date: Mon Feb 12 17:59:59 2007 Subject: [C2hs] patch applied (c2hs): Remove a missed FiniteMaps reference Message-ID: <20070212230543.GA15296@cvs.haskell.org> Thu Feb 8 14:50:17 PST 2007 Duncan Coutts * Remove a missed FiniteMaps reference M ./c2hs/c/C.hs -5 From duncan.coutts at worc.ox.ac.uk Mon Feb 12 18:05:44 2007 From: duncan.coutts at worc.ox.ac.uk (Duncan Coutts) Date: Mon Feb 12 18:00:00 2007 Subject: [C2hs] patch applied (c2hs): Remove unused stuff from Common and rename the remainder to Position Message-ID: <20070212230544.GA15310@cvs.haskell.org> Thu Feb 8 14:56:15 PST 2007 Duncan Coutts * Remove unused stuff from Common and rename the remainder to Position There wasn't much that was genuinely common. It was pretty much just the Position type and associated functions. ./base/admin/Common.hs -> ./base/general/Position.hs M ./base/errors/Errors.hs -1 +1 M ./base/general/Position.hs -71 +6 M ./base/state/State.hs -1 +1 M ./base/state/StateBase.hs -1 +1 M ./base/syms/Attributes.hs -1 +1 M ./base/syms/Idents.hs -1 +1 M ./base/syms/tests/Main.hs -1 +1 M ./base/syntax/Lexers.hs -1 +1 M ./base/syntax/tests/LexersTest.hs -1 +1 M ./base/syntax/tests/ParsersTest.hs -1 +1 M ./c2hs/c/CAST.hs -1 +1 M ./c2hs/c/CAttrs.hs -1 +1 M ./c2hs/c/CBuiltin.hs -1 +1 M ./c2hs/c/CLexer.x -1 +1 M ./c2hs/c/CNames.hs -1 +1 M ./c2hs/c/CParser.y -1 +1 M ./c2hs/c/CTrav.hs -1 +1 M ./c2hs/chs/CHS.hs -1 +1 M ./c2hs/chs/CHSLexer.hs -1 +1 M ./c2hs/gen/GBMonad.hs -1 +1 M ./c2hs/gen/GenBind.hs -1 +1 M ./c2hs/gen/GenHeader.hs -1 +1 From duncan.coutts at worc.ox.ac.uk Mon Feb 12 18:05:46 2007 From: duncan.coutts at worc.ox.ac.uk (Duncan Coutts) Date: Mon Feb 12 18:00:01 2007 Subject: [C2hs] patch applied (c2hs): Eliminate assertEnabled Message-ID: <20070212230546.GA15324@cvs.haskell.org> Thu Feb 8 15:02:15 PST 2007 Duncan Coutts * Eliminate assertEnabled It was no longer used. M ./base/admin/Config.hs -11 +1 M ./base/errors/Errors.hs -1 From duncan.coutts at worc.ox.ac.uk Mon Feb 12 18:05:47 2007 From: duncan.coutts at worc.ox.ac.uk (Duncan Coutts) Date: Mon Feb 12 18:00:03 2007 Subject: [C2hs] patch applied (c2hs): Remove the Utils module Message-ID: <20070212230547.GA15338@cvs.haskell.org> Thu Feb 8 15:03:46 PST 2007 Duncan Coutts * Remove the Utils module Most functions were not used. Move the couple functions that are used into the modules that use them. M ./base/errors/Errors.hs +7 R ./base/general/Utils.hs M ./base/state/State.hs +1 M ./c2hs/c/CNames.hs +6 M ./c2hs/gen/GenBind.hs -2 +13 From duncan.coutts at worc.ox.ac.uk Mon Feb 12 18:05:48 2007 From: duncan.coutts at worc.ox.ac.uk (Duncan Coutts) Date: Mon Feb 12 18:00:05 2007 Subject: [C2hs] patch applied (c2hs): Remove FileOps as it's not used. Message-ID: <20070212230548.GA15352@cvs.haskell.org> Thu Feb 8 15:12:47 PST 2007 Duncan Coutts * Remove FileOps as it's not used. R ./base/general/FileOps.hs M ./base/state/CIO.hs -15 +1 M ./c2hs/toplevel/Main.hs -1 +1 From duncan.coutts at worc.ox.ac.uk Mon Feb 12 18:05:49 2007 From: duncan.coutts at worc.ox.ac.uk (Duncan Coutts) Date: Mon Feb 12 18:00:05 2007 Subject: [C2hs] patch applied (c2hs): Remove missed uses of Common module Message-ID: <20070212230549.GA15366@cvs.haskell.org> Thu Feb 8 15:16:01 PST 2007 Duncan Coutts * Remove missed uses of Common module M ./base/state/StateTrans.hs -1 M ./c2hs/c/C.hs -1 +1 M ./c2hs/toplevel/Main.hs -1 From duncan.coutts at worc.ox.ac.uk Mon Feb 12 18:05:51 2007 From: duncan.coutts at worc.ox.ac.uk (Duncan Coutts) Date: Mon Feb 12 18:00:07 2007 Subject: [C2hs] patch applied (c2hs): Remove three unused modules Message-ID: <20070212230551.GA15382@cvs.haskell.org> Thu Feb 8 15:18:20 PST 2007 Duncan Coutts * Remove three unused modules R ./base/state/DynArrays.hs R ./base/syntax/ParserMonad.hs R ./base/syntax/Parsers.hs From duncan.coutts at worc.ox.ac.uk Mon Feb 12 18:05:52 2007 From: duncan.coutts at worc.ox.ac.uk (Duncan Coutts) Date: Mon Feb 12 18:00:08 2007 Subject: [C2hs] patch applied (c2hs): Remove several more unused modules Message-ID: <20070212230552.GA15396@cvs.haskell.org> Thu Feb 8 15:20:02 PST 2007 Duncan Coutts * Remove several more unused modules Sets and Marks were not being used. The GetOpt from the standard libs is now being used. R ./base/general/GetOpt.hs R ./base/general/Sets.hs R ./base/graphs/ R ./base/graphs/Marks.hs R ./base/sysdep/ From duncan.coutts at worc.ox.ac.uk Mon Feb 12 18:05:53 2007 From: duncan.coutts at worc.ox.ac.uk (Duncan Coutts) Date: Mon Feb 12 18:00:09 2007 Subject: [C2hs] patch applied (c2hs): Use the standard Text.PrettyPrint.HughesPJ and remove the bundled version Message-ID: <20070212230553.GA15410@cvs.haskell.org> Fri Feb 9 02:18:51 PST 2007 Duncan Coutts * Use the standard Text.PrettyPrint.HughesPJ and remove the bundled version R ./base/syntax/Pretty.hs M ./c2hs/c/CPretty.hs -2 +10 From duncan.coutts at worc.ox.ac.uk Mon Feb 12 18:05:55 2007 From: duncan.coutts at worc.ox.ac.uk (Duncan Coutts) Date: Mon Feb 12 18:00:10 2007 Subject: [C2hs] patch applied (c2hs): Remove unused imports and commented out code Message-ID: <20070212230555.GA15424@cvs.haskell.org> Fri Feb 9 02:20:17 PST 2007 Duncan Coutts * Remove unused imports and commented out code M ./base/state/State.hs -2 +1 M ./base/state/StateTrans.hs -19 From duncan.coutts at worc.ox.ac.uk Mon Feb 12 18:05:56 2007 From: duncan.coutts at worc.ox.ac.uk (Duncan Coutts) Date: Mon Feb 12 18:00:12 2007 Subject: [C2hs] patch applied (c2hs): Remove last few uses of Utils module Message-ID: <20070212230556.GA15438@cvs.haskell.org> Fri Feb 9 02:25:12 PST 2007 Duncan Coutts * Remove last few uses of Utils module M ./base/errors/Errors.hs -1 M ./base/state/State.hs -1 M ./c2hs/c/CNames.hs -2 +1 M ./c2hs/c/CParser.y -1 M ./c2hs/gen/GenBind.hs -1 From duncan.coutts at worc.ox.ac.uk Mon Feb 12 18:05:57 2007 From: duncan.coutts at worc.ox.ac.uk (Duncan Coutts) Date: Mon Feb 12 18:00:13 2007 Subject: [C2hs] patch applied (c2hs): Remove unused tmpdir Message-ID: <20070212230557.GA15452@cvs.haskell.org> Fri Feb 9 02:26:52 PST 2007 Duncan Coutts * Remove unused tmpdir M ./c2hs/toplevel/C2HSConfig.hs.in -7 +1 M ./c2hs/toplevel/Main.hs -1 +1 From duncan.coutts at worc.ox.ac.uk Mon Feb 12 18:05:59 2007 From: duncan.coutts at worc.ox.ac.uk (Duncan Coutts) Date: Mon Feb 12 18:00:16 2007 Subject: [C2hs] patch applied (c2hs): Update the .cabal file with the removed modules Message-ID: <20070212230559.GA15466@cvs.haskell.org> Fri Feb 9 02:41:46 PST 2007 Duncan Coutts * Update the .cabal file with the removed modules M ./c2hs.cabal -12 +1 From duncan.coutts at worc.ox.ac.uk Mon Feb 12 18:06:00 2007 From: duncan.coutts at worc.ox.ac.uk (Duncan Coutts) Date: Mon Feb 12 18:00:16 2007 Subject: [C2hs] patch applied (c2hs): Use Cabal' s mechanism to find the datadir rather than our own Message-ID: <20070212230600.GA15481@cvs.haskell.org> Fri Feb 9 02:42:38 PST 2007 Duncan Coutts * Use Cabal's mechanism to find the datadir rather than our own So remove the --data flag and the wrapper scripts. Cabal now installs the C2HS.hs file. Also means the Setup.hs no longer needs to install the wrapper script or C2HS.hs. ./c2hs/lib/C2HS.hs -> ./C2HS.hs M ./Setup.hs -26 +1 M ./c2hs.cabal +2 R ./c2hs/c2hs-inplace.in R ./c2hs/c2hs.in M ./c2hs/state/Switches.hs -2 M ./c2hs/toplevel/Main.hs -14 +3 R ./postInst.sh.in From duncan.coutts at worc.ox.ac.uk Mon Feb 12 18:06:01 2007 From: duncan.coutts at worc.ox.ac.uk (Duncan Coutts) Date: Mon Feb 12 18:00:18 2007 Subject: [C2hs] patch applied (c2hs): Eliminate need for ./configure Message-ID: <20070212230601.GA15497@cvs.haskell.org> Fri Feb 9 04:45:07 PST 2007 Duncan Coutts * Eliminate need for ./configure Move the version nick and date into the Version module directly (We might be able to get those into the cabal-generated module in future to make it more easily maintainable) Instead of using autoconf's host spec, use the equivalent from System.Info.(arch,os). Add host info for x86-64. Remove unused datadir var. ./c2hs/toplevel/C2HSConfig.hs.in -> ./c2hs/toplevel/C2HSConfig.hs ./c2hs/toplevel/Version.hs.in -> ./c2hs/toplevel/Version.hs M ./c2hs/toplevel/C2HSConfig.hs -14 +13 M ./c2hs/toplevel/Main.hs -4 +1 M ./c2hs/toplevel/Version.hs -7 +7 M ./configure.in -12 +1 From duncan.coutts at worc.ox.ac.uk Mon Feb 12 18:06:03 2007 From: duncan.coutts at worc.ox.ac.uk (Duncan Coutts) Date: Mon Feb 12 18:00:19 2007 Subject: [C2hs] patch applied (c2hs): Tidy up .cabal file Message-ID: <20070212230603.GA15511@cvs.haskell.org> Fri Feb 9 04:48:58 PST 2007 Duncan Coutts * Tidy up .cabal file M ./c2hs.cabal -3 +2 From duncan.coutts at worc.ox.ac.uk Mon Feb 12 18:06:04 2007 From: duncan.coutts at worc.ox.ac.uk (Duncan Coutts) Date: Mon Feb 12 18:00:21 2007 Subject: [C2hs] patch applied (c2hs): Make Position a proper (though not abstract) type Message-ID: <20070212230604.GA15525@cvs.haskell.org> Fri Feb 9 05:13:08 PST 2007 Duncan Coutts * Make Position a proper (though not abstract) type Also strictify and unpack the row and col numbers M ./base/errors/Errors.hs -2 +2 M ./base/general/Position.hs -26 +31 M ./base/syntax/Lexers.hs -8 +8 M ./c2hs/c/C.hs -2 +2 M ./c2hs/c/CLexer.x -5 +5 M ./c2hs/chs/CHS.hs -3 +3 M ./c2hs/chs/CHSLexer.hs -3 +3 From duncan.coutts at worc.ox.ac.uk Mon Feb 12 18:06:05 2007 From: duncan.coutts at worc.ox.ac.uk (Duncan Coutts) Date: Mon Feb 12 18:00:22 2007 Subject: [C2hs] patch applied (c2hs): Make the token constructors stricter Message-ID: <20070212230605.GA15539@cvs.haskell.org> Fri Feb 9 05:15:17 PST 2007 Duncan Coutts * Make the token constructors stricter Both in their semantic content and the position M ./c2hs/c/CLexer.x -89 +89 From duncan.coutts at worc.ox.ac.uk Mon Feb 12 18:06:07 2007 From: duncan.coutts at worc.ox.ac.uk (Duncan Coutts) Date: Mon Feb 12 18:00:24 2007 Subject: [C2hs] patch applied (c2hs): Remove the token number comments in the parser. Message-ID: <20070212230607.GA15553@cvs.haskell.org> Fri Feb 9 05:16:12 PST 2007 Duncan Coutts * Remove the token number comments in the parser. They were not accurate anyway. M ./c2hs/c/CParser.y -89 +89 From duncan.coutts at worc.ox.ac.uk Sun Feb 18 20:43:42 2007 From: duncan.coutts at worc.ox.ac.uk (Duncan Coutts) Date: Sun Feb 18 20:37:35 2007 Subject: [C2hs] a new C parser Message-ID: <1171849422.1018.164.camel@localhost> All, I posted some time ago about a plan to make a new C parser that would correctly deal with the typedef problem and then to try some automated testing by making a gcc wrapper. I've now mostly done that. Currently I can parse almost all of the Linux kernel, glibc, GNU binutils, and the C code produced by ghc when it compiles itself. I can also parse most of the artificial nasty typedef examples: typedef int A, B(A); I think I correctly deal with scope of typedef names and typedef names being shadowed by local variable declarations: typedef int A; A main () { A A; -- declares local var A as having type A A = A + 1; -- A now refers to the local var, not the type; } A x; -- end of scope, A reverts to referring to the typedef name. A slightly more tricky example of the same thing is where function parameters redeclare names: typedef int A; A main (A A) { A = A + 1; -- A now refers to the local var, not the type; } An example I don't deal with yet is: typedef int A; int A = sizeof(A); that is we shadow the typename A with the local var A, so the A in the initialiser should now refer to the local var not the typename. However currently the parser only does the shadowing after the ';', not after the '=' as would be required. This is because the production rule looks like: declaring_list : declaration_specifier declarator initializer_opt {% do doDeclIdent $1 $2 return $ CDecl $1 [($2, $3, Nothing)]) } where as we'd really like to say: declaring_list : declaration_specifier declarator {% doDeclIdent $1 $2 } initializer_opt { CDecl $1 [($2, $4, Nothing)]) } that is to modify the typedef set immediately after the declarator and before parsing the initializer. This is exactly how people do it with YACC but unfortunately happy does not support this form where actions are interspersed in the production. It's probably possible to refactor the grammar to do this with happy, though it's not totally obvious to me how. Dealing with '__attribute__'s: ============================== GNU C attributes are a real pain. They seem to be able to appear almost anywhere in a declaration. However adding them to the grammar has to be done very carefully to avoid introducing ambiguities (ie shift/reduce or reduce/reduce conflicts). So for the moment I've taken the approach of not parsing them at all but recognising and ignoring attributes in the lexer and not passing them on to the parser at all. This effectively treats attributes as whitespace and they can appear anywhere. This isn't a long term solution because eventually we'll need to recognise some attributes because they can affect structure layout and we need that for accurately calculating the size of types and offsets of structure members. C99 extensions: =============== I'm not sure I've got 100% coverage of C99 features, but at least I cover the ones used in the code I've seen so far. These are just the ones that I can remember adding: * mixed declarations and statements in a compound statement * compound literals * for statements with declarations: for (int n = 0; n < m; n++) {}; * wide character character and string literals * unsigned long long numeric literals * _Bool and _Complex basic types GNU extensions: =============== Again, just some of the GNU extensions that I can remember adding to the new parser compared to the extensions the previous parser recognised: * anonymous structs and unions inside other structs/unions. * thread local storage qualifier * old style and array range compound literal member designator * case ranges * computed gotos and address of labels * conditional operator with missing then part: "x ? : y" * empty structs * allow redundant ';' in several places * asm expressions * several built-in 'functions' that take type names as parameters Various of these extensions require extensions in the AST. Testing: ======== I've made a cc-wrapper program that you can use as if it were gcc. It calls gcc with the same args as it was called with. Then if it can figure out what the gcc args meant and it looks like gcc was being asked to compile a .c file then it tries to pre-process and parse the same .c file. It outputs results to a log file and puts detailed reports on parse failures into separate files (in a directory specified via an environment variable). So (in theory at least) one ought to be able to compile vast amounts of C code and find out which extensions are really used and break the parser. At the moment the it doesn't check that it correctly parses the code, only that it parses it at all. Extra checks as I mentioned before could include checking if the sizes of types and offsets of structure members computed by c2hs match those computed by gcc. Another test would be to pretty-print the C code again and see if gcc can still parse it and if it produces the same object code (that'd require fully parsing attributes). As I said, I've been testing this on the linux kernel (using allyesconfig) glibc and ghc. Last time I tried it against linux it failed on 7 out of 4406 .c files. All those failures were due to the same bug which I think I've now fixed. In glibc, a few files fail to parse due to their use of nested function declarations which is a rarely used GNU extension that I've not implemented yet. Plan: ===== So there are a few things to do, one is to continue testing large amounts of C code. I intend to include cc-wrapper in c2hs (or perhaps do it as a c2hs mode via a flag) so that users can do their own testing and provide a way to give detailed bug reports. I haven't checked yet that the new C parser doesn't break things when doing the full .chs -> .hs translations. Also, I've not extended the guts of c2hs to understand any of these new language extensions so trying to bind things which really use them may not work. Then I need to start merging the new parser into the darcs version of c2hs. I'll try and do this as a series of small understandable patches. If anyone wants to play with the code I've got right now rather than wait for things to get merged then do say. I could put up a tarball. Duncan From duncan.coutts at worc.ox.ac.uk Sun Feb 18 20:44:18 2007 From: duncan.coutts at worc.ox.ac.uk (Duncan Coutts) Date: Sun Feb 18 20:38:12 2007 Subject: [C2hs] Re: [Haskell-cafe] c2hs failed to process __extension__ In-Reply-To: <1170075335.16642.199.camel@localhost> References: <1170075335.16642.199.camel@localhost> Message-ID: <1171849458.1018.166.camel@localhost> On Mon, 2007-01-29 at 12:55 +0000, Duncan Coutts wrote: > On Mon, 2007-01-29 at 21:25 +0900, ?? ? wrote: > > > c2hs fails to process __extension__? > > > > From the changelog, c2hs now can understand __extension. > > Is this an exceptional case? static __inline __uint32_t __bswap32(__uint32_t _x) { return (__extension__ (({ register __uint32_t __X = (_x); __asm ("bswap %0" : "+r" (__X)); __X; }))); } My new c2hs C parser can parse your example and indeed the whole of that Node.i file that you posted. The GNU C extensions required were not just the __extension__ expression but also compound statements as expressions. The asm statement extension was already implemented. So when I get this new parser into the mainline c2hs you should try again. Duncan From chak at cse.unsw.edu.au Sun Feb 18 21:08:21 2007 From: chak at cse.unsw.edu.au (Manuel M T Chakravarty) Date: Sun Feb 18 21:03:09 2007 Subject: [C2hs] a new C parser In-Reply-To: <1171849422.1018.164.camel@localhost> References: <1171849422.1018.164.camel@localhost> Message-ID: <1171850901.29909.11.camel@AttitudeAdjuster.localdomain> Duncan Coutts: > I posted some time ago about a plan to make a new C parser that would > correctly deal with the typedef problem and then to try some automated > testing by making a gcc wrapper. I've now mostly done that. > > Currently I can parse almost all of the Linux kernel, glibc, GNU > binutils, and the C code produced by ghc when it compiles itself. [..] > So there are a few things to do, one is to continue testing large > amounts of C code. I intend to include cc-wrapper in c2hs (or perhaps do > it as a c2hs mode via a flag) so that users can do their own testing and > provide a way to give detailed bug reports. > > I haven't checked yet that the new C parser doesn't break things when > doing the full .chs -> .hs translations. Also, I've not extended the > guts of c2hs to understand any of these new language extensions so > trying to bind things which really use them may not work. > > Then I need to start merging the new parser into the darcs version of > c2hs. I'll try and do this as a series of small understandable patches. Wow, Duncan, you are a hero!! Thanks for all that fantastic work. Manuel From jelmer at samba.org Sun Feb 18 21:23:38 2007 From: jelmer at samba.org (Jelmer Vernooij) Date: Sun Feb 18 21:18:28 2007 Subject: [C2hs] a new C parser In-Reply-To: <1171849422.1018.164.camel@localhost> References: <1171849422.1018.164.camel@localhost> Message-ID: <1171851818.16678.11.camel@ganieda.lan.vernstok.nl> On Mon, 2007-02-19 at 01:43 +0000, Duncan Coutts wrote: > I posted some time ago about a plan to make a new C parser that would > correctly deal with the typedef problem and then to try some automated > testing by making a gcc wrapper. I've now mostly done that. > > Currently I can parse almost all of the Linux kernel, glibc, GNU > binutils, and the C code produced by ghc when it compiles itself. [...] > Then I need to start merging the new parser into the darcs version of > c2hs. I'll try and do this as a series of small understandable patches. > > If anyone wants to play with the code I've got right now rather than > wait for things to get merged then do say. I could put up a tarball. That's very nice! Problems with the current C parser are blocking me from actually using c2hs at the moment (and I failed at my attempts to fix it). If there's anything I could test, please let me know. Cheers, Jelmer -- Jelmer Vernooij - http://samba.org/~jelmer/ -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://www.haskell.org/pipermail/c2hs/attachments/20070219/1e6909ae/attachment.bin From duncan.coutts at worc.ox.ac.uk Mon Feb 19 07:51:58 2007 From: duncan.coutts at worc.ox.ac.uk (Duncan Coutts) Date: Mon Feb 19 07:45:49 2007 Subject: [C2hs] a new C parser In-Reply-To: <1171851818.16678.11.camel@ganieda.lan.vernstok.nl> References: <1171849422.1018.164.camel@localhost> <1171851818.16678.11.camel@ganieda.lan.vernstok.nl> Message-ID: <1171889518.1018.186.camel@localhost> On Mon, 2007-02-19 at 03:23 +0100, Jelmer Vernooij wrote: > On Mon, 2007-02-19 at 01:43 +0000, Duncan Coutts wrote: > > I posted some time ago about a plan to make a new C parser that would > > correctly deal with the typedef problem and then to try some automated > > testing by making a gcc wrapper. I've now mostly done that. > > > > Currently I can parse almost all of the Linux kernel, glibc, GNU > > binutils, and the C code produced by ghc when it compiles itself. > [...] > > Then I need to start merging the new parser into the darcs version of > > c2hs. I'll try and do this as a series of small understandable patches. > > > > If anyone wants to play with the code I've got right now rather than > > wait for things to get merged then do say. I could put up a tarball. > That's very nice! Problems with the current C parser are blocking me > from actually using c2hs at the moment (and I failed at my attempts to > fix it). If there's anything I could test, please let me know. So I've been compiling various packages with this cc-wrapper. I could do with some help on that. Mind you, Gentoo makes this fairly easy to do. If you've got other sources you want it to work with then now's a good time to try. Here's the tarball of my current tree: http://haskell.org/~duncan/c2hs/c2hs-newparser.tar.gz (Tarball unpacks to just c2hs/) to test: $ cd c2hs/c2hs/c/tests $ make cc-wrapper $ mkdir /tmp/c2hs $ export C2HS_CC_LOGDIR=/tmp/c2hs then invoke cc-wrapper as if it were gcc. look in $C2HS_CC_LOGDIR for a .log file and error .report files. The .report files are self contained, that is they should have all the info necessary for someone else to reproduce the failure (since it contains the full pre-processed source). An improved cc-wrapper script would be helpful though. I wrote it in Haskell however there's a problem there. GHC's I/O libs always put all file descriptors into non-blocking mode and since the blocking/non-blocking status on open file descriptors is shared between processes (silly design imho) this can cause problems. For some compilations (especially when compiling glibc for some reason) I get this error from cc1 about stdout being "temporarily unavailable". I think the solution is to use a bash (or whatever) script to capture the gcc args, parse them and pass them onto the Haskell C parser prog. Patches welcome :-) Duncan From duncan.coutts at worc.ox.ac.uk Mon Feb 19 15:46:27 2007 From: duncan.coutts at worc.ox.ac.uk (Duncan Coutts) Date: Mon Feb 19 15:40:20 2007 Subject: [C2hs] patch applied (c2hs): Split tokens and parser monad out of Lexer.x into their own modules Message-ID: <20070219204627.GA20348@cvs.haskell.org> Mon Feb 12 04:20:37 PST 2007 Duncan Coutts * Split tokens and parser monad out of Lexer.x into their own modules This allows the parser monad to be made a little more abstract. M ./c2hs/c/CLexer.x -382 +4 A ./c2hs/c/CParserMonad.hs A ./c2hs/c/CTokens.hs From duncan.coutts at worc.ox.ac.uk Mon Feb 19 15:46:28 2007 From: duncan.coutts at worc.ox.ac.uk (Duncan Coutts) Date: Mon Feb 19 15:40:22 2007 Subject: [C2hs] patch applied (c2hs): Rename parser state fields, change get/setInput type and tidy imports Message-ID: <20070219204628.GA20368@cvs.haskell.org> Mon Feb 12 10:59:51 PST 2007 Duncan Coutts * Rename parser state fields, change get/setInput type and tidy imports M ./c2hs/c/CLexer.x -12 +15 M ./c2hs/c/CParser.y -2 +3 M ./c2hs/c/CParserMonad.hs -24 +24 From duncan.coutts at worc.ox.ac.uk Mon Feb 19 15:46:29 2007 From: duncan.coutts at worc.ox.ac.uk (Duncan Coutts) Date: Mon Feb 19 15:40:23 2007 Subject: [C2hs] patch applied (c2hs): Add scope history and enterScope/leaveScope Message-ID: <20070219204629.GA20382@cvs.haskell.org> Mon Feb 12 11:00:42 PST 2007 Duncan Coutts * Add scope history and enterScope/leaveScope entering a scope pushes the current tyident set into a history stack leaving a scope pops that stack. M ./c2hs/c/CParserMonad.hs -1 +15 From duncan.coutts at worc.ox.ac.uk Mon Feb 19 15:46:31 2007 From: duncan.coutts at worc.ox.ac.uk (Duncan Coutts) Date: Mon Feb 19 15:40:25 2007 Subject: [C2hs] patch applied (c2hs): Rename vars and reorder exports in ParserMonad module Message-ID: <20070219204631.GA20396@cvs.haskell.org> Mon Feb 19 12:33:58 PST 2007 Duncan Coutts * Rename vars and reorder exports in ParserMonad module M ./c2hs/c/CParserMonad.hs -12 +12 From duncan.coutts at worc.ox.ac.uk Mon Feb 19 15:46:32 2007 From: duncan.coutts at worc.ox.ac.uk (Duncan Coutts) Date: Mon Feb 19 15:40:26 2007 Subject: [C2hs] patch applied (c2hs): Add shadowTypedef operation to parser monad Message-ID: <20070219204632.GA20410@cvs.haskell.org> Mon Feb 19 12:35:13 PST 2007 Duncan Coutts * Add shadowTypedef operation to parser monad Local vars in inner scopes can shadow type names so we need to be able to adjust the current typedef identifier set accordingly (ie delete identifiers from the typedef ident set). For example: typedef int X; /* X declared as a type name */ void foo() { int X; X = X +1 } /* X redeclared as a local var */ M ./c2hs/c/CParserMonad.hs -1 +11 From duncan.coutts at worc.ox.ac.uk Tue Feb 20 05:25:23 2007 From: duncan.coutts at worc.ox.ac.uk (Duncan Coutts) Date: Tue Feb 20 05:19:15 2007 Subject: [C2hs] patch applied (c2hs): Ignore all __attribute__ expressions in the lexer Message-ID: <20070220102523.GA2130@cvs.haskell.org> Mon Feb 19 14:45:49 PST 2007 Duncan Coutts * Ignore all __attribute__ expressions in the lexer GNU __attribute__ expressions take the form of: __attribute__ '(' '(' ... ')' ')' where the stuff in the middle may also have nested '(' ')' brackets. So upon getting an __attribute__ token, the lexer now skips tokens counting bracket nesting depth and stops when it's seen the full attribute args. So the parser now never sees attributes which is good because they greatly complicate the grammar of declarations. On the other hand it's bad because later on we might need the info from some attributes. So we'll likely have to revisit this issue. M ./c2hs/c/CLexer.x -2 +14 From duncan.coutts at worc.ox.ac.uk Tue Feb 20 05:25:24 2007 From: duncan.coutts at worc.ox.ac.uk (Duncan Coutts) Date: Tue Feb 20 05:19:16 2007 Subject: [C2hs] patch applied (c2hs): Oops, missing ',' Message-ID: <20070220102524.GA2150@cvs.haskell.org> Tue Feb 20 01:39:40 PST 2007 Duncan Coutts * Oops, missing ',' M ./c2hs/c/CParserMonad.hs -1 +1 From duncan.coutts at worc.ox.ac.uk Tue Feb 20 05:25:25 2007 From: duncan.coutts at worc.ox.ac.uk (Duncan Coutts) Date: Tue Feb 20 05:19:17 2007 Subject: [C2hs] patch applied (c2hs): Dump perfect attribute order compatability with the previous parser Message-ID: <20070220102525.GA2165@cvs.haskell.org> Tue Feb 20 01:40:55 PST 2007 Duncan Coutts * Dump perfect attribute order compatability with the previous parser We can't preserve this anyway so no need for keeping the code that tries to do so. M ./c2hs/c/CParser.y -11 +7 From nubgames at gmail.com Tue Feb 20 20:40:47 2007 From: nubgames at gmail.com (Eric Sessoms) Date: Tue Feb 20 20:34:37 2007 Subject: [C2hs] help a newbie? Message-ID: <9d4cb2970702201740o6c90db0fre3856e7748aa16da@mail.gmail.com> I'm working on a problem that looks, to my eye, identical to the example in section 2.10 of the documentation http://www.cse.unsw.edu.au/~chak/haskell/c2hs/docu/c2hs-2.html#ss2.10 --- cut here --- module Test where #c typedef struct { int dummy; } fitsfile; int ffclos(fitsfile *fptr, int *status); #endc {# pointer *fitsfile as FitsFile newtype #} {# fun ffclos { `FitsFile', `Int' } -> `Int' #} --- cut here -- But running c2hs I get an error message that I haven't been able to make use of: Test.chs:13: (column 17) [ERROR] >>> Missing "in" marshaller! There is no default marshaller for this combination of Haskell and C type: Haskell type: FitsFile C type : (FitsFile) I'm particularly puzzled because the corresponding "open" function (much more complex, omitted for brevity) that marshals a **fitsfile doesn't give me any problems. Suggestions? Thanks much, Eric From duncan.coutts at worc.ox.ac.uk Thu Feb 22 15:57:07 2007 From: duncan.coutts at worc.ox.ac.uk (Duncan Coutts) Date: Thu Feb 22 15:51:54 2007 Subject: [C2hs] help a newbie? In-Reply-To: <9d4cb2970702201740o6c90db0fre3856e7748aa16da@mail.gmail.com> References: <9d4cb2970702201740o6c90db0fre3856e7748aa16da@mail.gmail.com> Message-ID: <1172177827.1018.249.camel@localhost> On Tue, 2007-02-20 at 20:40 -0500, Eric Sessoms wrote: > I'm working on a problem that looks, to my eye, identical to the > example in section 2.10 of the documentation > http://www.cse.unsw.edu.au/~chak/haskell/c2hs/docu/c2hs-2.html#ss2.10 > > --- cut here --- > module Test where > > #c > typedef struct { > int dummy; > } fitsfile; > > int ffclos(fitsfile *fptr, int *status); > #endc > > {# pointer *fitsfile as FitsFile newtype #} > > {# fun ffclos { `FitsFile', `Int' } -> `Int' #} > --- cut here -- > > But running c2hs I get an error message that I haven't been able to make use of: > > Test.chs:13: (column 17) [ERROR] > >>> Missing "in" marshaller! > There is no default marshaller for this combination of Haskell and C type: > Haskell type: FitsFile > C type : (FitsFile) I'm not sure if c2hs is supposed to work out the default marshaler automatically, you'd have to ask Manuel, but it does work if you specify the marshaler explicitly: {# fun ffclos { withFitsFile `FitsFile', `Int' } -> `Int' #} Duncan From duncan.coutts at worc.ox.ac.uk Thu Feb 22 19:39:10 2007 From: duncan.coutts at worc.ox.ac.uk (Duncan Coutts) Date: Thu Feb 22 19:32:54 2007 Subject: [C2hs] patch applied (c2hs): Remove GNU C attributes from the grammar Message-ID: <20070223003910.GA13460@cvs.haskell.org> Thu Feb 22 14:30:41 PST 2007 Duncan Coutts * Remove GNU C attributes from the grammar since they are now recognised and discarded by the lexer. M ./c2hs/c/CParser.y -16 +13 From duncan.coutts at worc.ox.ac.uk Thu Feb 22 19:39:11 2007 From: duncan.coutts at worc.ox.ac.uk (Duncan Coutts) Date: Thu Feb 22 19:32:55 2007 Subject: [C2hs] patch applied (c2hs): Use section names from the C99 report rather than from K&R Message-ID: <20070223003911.GA13480@cvs.haskell.org> Thu Feb 22 14:42:21 PST 2007 Duncan Coutts * Use section names from the C99 report rather than from K&R Many grammar productions have notes which refer to the C spec, since we're targeting C99 now make them refer to the section numbers in the C99 report rather than from K&R. M ./c2hs/c/CParser.y -42 +42 From duncan.coutts at worc.ox.ac.uk Thu Feb 22 19:39:13 2007 From: duncan.coutts at worc.ox.ac.uk (Duncan Coutts) Date: Thu Feb 22 19:32:56 2007 Subject: [C2hs] patch applied (c2hs): Oops, missed a few K&R -> C99 section number changes Message-ID: <20070223003913.GA13494@cvs.haskell.org> Thu Feb 22 15:00:47 PST 2007 Duncan Coutts * Oops, missed a few K&R -> C99 section number changes M ./c2hs/c/CParser.y -3 +3 From duncan.coutts at worc.ox.ac.uk Thu Feb 22 19:39:14 2007 From: duncan.coutts at worc.ox.ac.uk (Duncan Coutts) Date: Thu Feb 22 19:32:58 2007 Subject: [C2hs] patch applied (c2hs): Use a newtype for building reversed lists Message-ID: <20070223003914.GA13511@cvs.haskell.org> Thu Feb 22 16:21:30 PST 2007 Duncan Coutts * Use a newtype for building reversed lists When building lists up in reverse it's rather easy to forget to reverse them when we get to the top level. So introduce a newtype and operations for reversed lists. This way the type will tell us where we've forgot to reverse (and there were one or two places) M ./c2hs/c/CParser.y -46 +77 From duncan.coutts at worc.ox.ac.uk Thu Feb 22 19:39:15 2007 From: duncan.coutts at worc.ox.ac.uk (Duncan Coutts) Date: Thu Feb 22 19:32:59 2007 Subject: [C2hs] patch applied (c2hs): Tidy up parsing of comma expressions Message-ID: <20070223003915.GA13525@cvs.haskell.org> Thu Feb 22 16:31:38 PST 2007 Duncan Coutts * Tidy up parsing of comma expressions M ./c2hs/c/CParser.y -7 +9 From duncan.coutts at worc.ox.ac.uk Thu Feb 22 19:39:16 2007 From: duncan.coutts at worc.ox.ac.uk (Duncan Coutts) Date: Thu Feb 22 19:33:01 2007 Subject: [C2hs] patch applied (c2hs): Parse string literals more in the spirit of C99 Message-ID: <20070223003916.GA13539@cvs.haskell.org> Thu Feb 22 16:36:45 PST 2007 Duncan Coutts * Parse string literals more in the spirit of C99 M ./c2hs/c/CParser.y -27 +23 From nubgames at gmail.com Thu Feb 22 19:56:00 2007 From: nubgames at gmail.com (Eric Sessoms) Date: Thu Feb 22 19:49:43 2007 Subject: [C2hs] help a newbie? In-Reply-To: <1172177827.1018.249.camel@localhost> References: <9d4cb2970702201740o6c90db0fre3856e7748aa16da@mail.gmail.com> <1172177827.1018.249.camel@localhost> Message-ID: <9d4cb2970702221656i1a691c9bu70c7cf4525200beb@mail.gmail.com> Thanks, Duncan. Related to this, for anyone who might run into the same problem, I've hit several other instances of c2hs refusing to marshal Haskell type "a" to C type "(a)" that I've worked around by specifying "id" for the marshal function. Go figure. :) At least it makes sense now that I've been living with it for a few days. Followup: is there any way to specify a default marshaller? For example, is there a way to specify in one place "always use withFitsFile* to marshal a FitsFile" (or similar)? Eric On 2/22/07, Duncan Coutts wrote: > On Tue, 2007-02-20 at 20:40 -0500, Eric Sessoms wrote: > > I'm working on a problem that looks, to my eye, identical to the > > example in section 2.10 of the documentation > > http://www.cse.unsw.edu.au/~chak/haskell/c2hs/docu/c2hs-2.html#ss2.10 > > > > --- cut here --- > > module Test where > > > > #c > > typedef struct { > > int dummy; > > } fitsfile; > > > > int ffclos(fitsfile *fptr, int *status); > > #endc > > > > {# pointer *fitsfile as FitsFile newtype #} > > > > {# fun ffclos { `FitsFile', `Int' } -> `Int' #} > > --- cut here -- > > > > But running c2hs I get an error message that I haven't been able to make use of: > > > > Test.chs:13: (column 17) [ERROR] > > >>> Missing "in" marshaller! > > There is no default marshaller for this combination of Haskell and C type: > > Haskell type: FitsFile > > C type : (FitsFile) > > I'm not sure if c2hs is supposed to work out the default marshaler > automatically, you'd have to ask Manuel, but it does work if you specify > the marshaler explicitly: > > {# fun ffclos { withFitsFile `FitsFile', `Int' } -> `Int' #} > > Duncan > > -- not about to see your light From chak at cse.unsw.edu.au Thu Feb 22 23:04:02 2007 From: chak at cse.unsw.edu.au (Manuel M T Chakravarty) Date: Thu Feb 22 22:58:38 2007 Subject: [C2hs] help a newbie? In-Reply-To: <1172177827.1018.249.camel@localhost> References: <9d4cb2970702201740o6c90db0fre3856e7748aa16da@mail.gmail.com> <1172177827.1018.249.camel@localhost> Message-ID: <1172203442.20679.120.camel@AttitudeAdjuster.localdomain> Duncan Coutts: > On Tue, 2007-02-20 at 20:40 -0500, Eric Sessoms wrote: > > I'm working on a problem that looks, to my eye, identical to the > > example in section 2.10 of the documentation > > http://www.cse.unsw.edu.au/~chak/haskell/c2hs/docu/c2hs-2.html#ss2.10 > > > > --- cut here --- > > module Test where > > > > #c > > typedef struct { > > int dummy; > > } fitsfile; > > > > int ffclos(fitsfile *fptr, int *status); > > #endc > > > > {# pointer *fitsfile as FitsFile newtype #} > > > > {# fun ffclos { `FitsFile', `Int' } -> `Int' #} > > --- cut here -- > > > > But running c2hs I get an error message that I haven't been able to make use of: > > > > Test.chs:13: (column 17) [ERROR] > > >>> Missing "in" marshaller! > > There is no default marshaller for this combination of Haskell and C type: > > Haskell type: FitsFile > > C type : (FitsFile) > > I'm not sure if c2hs is supposed to work out the default marshaler > automatically, you'd have to ask Manuel, but it does work if you specify > the marshaler explicitly: > > {# fun ffclos { withFitsFile `FitsFile', `Int' } -> `Int' #} You have to specify the marshaller explicitly. It would be nice to get default marshallers for types defined by pointer hooks, but that's currently not implemented. Manuel From duncan.coutts at worc.ox.ac.uk Fri Feb 23 05:53:06 2007 From: duncan.coutts at worc.ox.ac.uk (Duncan Coutts) Date: Fri Feb 23 05:46:49 2007 Subject: [C2hs] patch applied (c2hs): Use an optional expression production to simplify a couple productions. Message-ID: <20070223105306.GA24691@cvs.haskell.org> Fri Feb 23 02:34:28 PST 2007 Duncan Coutts * Use an optional expression production to simplify a couple productions. M ./c2hs/c/CParser.y -13 +10 From duncan.coutts at worc.ox.ac.uk Fri Feb 23 05:53:08 2007 From: duncan.coutts at worc.ox.ac.uk (Duncan Coutts) Date: Fri Feb 23 05:46:51 2007 Subject: [C2hs] patch applied (c2hs): Expand the enumerator_list and eliminate the enumerator_list_ production Message-ID: <20070223105308.GA24711@cvs.haskell.org> Fri Feb 23 02:38:16 PST 2007 Duncan Coutts * Expand the enumerator_list and eliminate the enumerator_list_ production This is more in keeping with the standard C99 grammar. M ./c2hs/c/CParser.y -10 +8 From duncan.coutts at worc.ox.ac.uk Fri Feb 23 05:53:09 2007 From: duncan.coutts at worc.ox.ac.uk (Duncan Coutts) Date: Fri Feb 23 05:46:52 2007 Subject: [C2hs] patch applied (c2hs): Parse C99 compound statements with mixed declarations and statements Message-ID: <20070223105309.GA24725@cvs.haskell.org> Fri Feb 23 02:47:40 PST 2007 Duncan Coutts * Parse C99 compound statements with mixed declarations and statements Rather than the K&R/C89 requirement that declarations precede statements. The AST is now a list of eithers rather than two lists. Note also that we're now saving and restoring the set of typedef'ed identifiers as we enter and leave the scope of the compound statement. M ./c2hs/c/CParser.y -15 +30 From chak at cse.unsw.edu.au Sun Feb 25 20:07:28 2007 From: chak at cse.unsw.edu.au (Manuel M T Chakravarty) Date: Sun Feb 25 20:01:53 2007 Subject: [C2hs] help a newbie? In-Reply-To: <9d4cb2970702221656i1a691c9bu70c7cf4525200beb@mail.gmail.com> References: <9d4cb2970702201740o6c90db0fre3856e7748aa16da@mail.gmail.com> <1172177827.1018.249.camel@localhost> <9d4cb2970702221656i1a691c9bu70c7cf4525200beb@mail.gmail.com> Message-ID: <1172452048.20795.9.camel@AttitudeAdjuster.localdomain> Eric Sessoms: > Followup: is there any way to specify a default marshaller? For > example, is there a way to specify in one place "always use > withFitsFile* to marshal a FitsFile" (or similar)? That would be useful, but like with default marshallers for pointer hooks, it is not currently implemented. Manuel > On 2/22/07, Duncan Coutts wrote: > > On Tue, 2007-02-20 at 20:40 -0500, Eric Sessoms wrote: > > > I'm working on a problem that looks, to my eye, identical to the > > > example in section 2.10 of the documentation > > > http://www.cse.unsw.edu.au/~chak/haskell/c2hs/docu/c2hs-2.html#ss2.10 > > > > > > --- cut here --- > > > module Test where > > > > > > #c > > > typedef struct { > > > int dummy; > > > } fitsfile; > > > > > > int ffclos(fitsfile *fptr, int *status); > > > #endc > > > > > > {# pointer *fitsfile as FitsFile newtype #} > > > > > > {# fun ffclos { `FitsFile', `Int' } -> `Int' #} > > > --- cut here -- > > > > > > But running c2hs I get an error message that I haven't been able to make use of: > > > > > > Test.chs:13: (column 17) [ERROR] > > > >>> Missing "in" marshaller! > > > There is no default marshaller for this combination of Haskell and C type: > > > Haskell type: FitsFile > > > C type : (FitsFile) > > > > I'm not sure if c2hs is supposed to work out the default marshaler > > automatically, you'd have to ask Manuel, but it does work if you specify > > the marshaler explicitly: > > > > {# fun ffclos { withFitsFile `FitsFile', `Int' } -> `Int' #} > > > > Duncan > > > > > >