You can achieve this this with Doxygen[http://www.stack.nl/~dimitri/doxygen/]. Doxygen itself do much more than just that. Couldn't stop doing it since the day I learned about it.

Hello Xiaolei Zhu, As far i know, Doxygen is good for documentation. It also can be used for generating different diagram like class diagram, call diagram. Currently I am using doxygen to generate XML file from C code. But from a programmer viewpoint we need something concrete which can be utilized in code. Thank you.

I would take a look at cscope and/or exuberant ctags. Some google also turned up pycparser. These may not be the defacto ones - I'm just guessing. Is there no open C tokenizer / lexer?

http://cscope.sourceforge.net/

http://ctags.sourceforge.net/

https://github.com/eliben/pycparser

I suggest having a look a clang (http://clang.llvm.org/). I haven't used it on my own. But, this library has a nice modern design and is very modular. It is build to be also used for static code analysis, etc. besides being a compiler front end.

For a certain parts of C code, such as array definitions, you can write a regular expression. C code is in fact very strict and rules are actually quite simple. The only more complicated part that I can think of now are escaped characters in string elements added to arrays, but as for numbers (integers or floats) it shouldn't be too complicated.

@Milan

It is never a good idea to reinvent the wheel. If there is a library that does the job, there is no reason why you should write your own. A library will (usually) constantly evolve and fix any bugs. And with open source libraries you get all this constant update for free.

And parsing C is definitely not easy. There are so many corner cases that have to be taken care of that I would definitely advise against trying to write your own parser. At some point you will run into the corner cases and then you will fix your parser. After a while this will result in a really ugly beast of code. You will have to plan for all of this from the beginning to have clean code in the end. But, that takes up a lot of effort.

I recommend this other CIL, the "Infrastructure for C Program Analysis and Transformation".

http://www.cs.berkeley.edu/~necula/cil/

It has extensive support for all kinds of languages extensions from GCC, Intel C, Visual C and LLVM/Clang. The framework is very modular and extremely easy to use. Because it is written in a functional way adding new passes / filters to extract information is exceedingly easy.

You can use a scripting language, such as Perl, and parse the file looking all the possibilities:

- "//" for single-line comments.

- "/*" and "*/" for multi-line comments.

- doxygen ones "///", "/**", "/*!"

Or try this tool:

http://search.cpan.org/~rokr/String-Comments-Extract-0.023/lib/String/Comments/Extract.pm

Try coccinelle: http://coccinelle.lip6.fr

A really nice tool is clang-check, which is part of the llvm/clang compiler suite others have mentioned. Here's a handy way to invoke it, and capture useful output:

clang-check my_c_program.c -ast-dump -- | sed -e 's/^[\[[0-9;]*m//g'

This generates an annotated C or C++ abstract syntax tree. Every identifier node has information indicating what type of identifier it is, and a cross-tree reference to the declaration site. This tool gives much more information about a C program than simply its syntax tree; it provides much information about the static semantics of the program as well. A real winner!

Just in case you also need to do memory profiling and footprint stuff, I guess you'd really like Valgrind:

http://valgrind.org/

especially if you're under Linux. And, BTW, there is an old book C++ & XML that I guess can alleviate some of your pain, in case you're willing to move to C++:

http://www.amazon.co.uk/XML-New-Riders-Landmark-Series/dp/073571052X/ref=sr_1_1?ie=UTF8&qid=1390472080&sr=8-1&keywords=C%2B%2B+XML

If you are indeed ready to migrate to C++, the Boost libraries are your best friends:

http://www.boost.org/

HTH,

--Hossein

nice idea

i need an analysis tool to extract variables, data type strcutures from very big source code and calculate memory footprint

any idea ?

I tried to write an own parser which identifies all function declarations and their bodies. There were so many corner cases I ended up having a regular expression the length of which was much longer then my screen. Eventually it seems it does the job, but after not using it for months I forgot what part of that regular expression does what. Feels like a trap!