Discussion:
GDB-like debugger for template metaprograms
(too old to reply)
Abel Sinkovics
2014-11-26 20:22:18 UTC
Permalink
Hi,

Metashell 2.0.0 has a template metaprogram debugger with a GDB-like
interface (thanks to András Kucsma), which can be useful for Boost
developers. It can show you how your metaprogram is executed
step-by-step. You can set breakpoints, step into or over the
metafunction calls and inspect the "template metaprogramming stack
trace" at any point during this: the chain of recursive template
#include <boost/mpl/transform.hpp>
#include <boost/mpl/vector_c.hpp>
#include <boost/mpl/plus.hpp>
#include <boost/mpl/lambda.hpp>
#include <boost/mpl/int.hpp>
using namespace boost::mpl;
#msh mdb transform<vector_c<int, 1, 2, 3, 4, 5>, plus<_1, int_<1>>>::type
For help, type "help".
Metaprogram started
(mdb) rbreak ^boost::mpl::plus
Breakpoint "^boost::mpl::plus" will stop the execution on 25 locations
(mdb) continue
Breakpoint "^boost::mpl::plus" reached
boost::mpl::plus<mpl_::integral_c<int, 1>, mpl_::int_<1>, mpl_::na,
mpl_::na, mpl_::na> (Memoization)
(mdb) bt
#0 boost::mpl::plus<mpl_::integral_c<int, 1>, mpl_::int_<1>, mpl_::na,
mpl_::na, mpl_::na> (Memoization)
#1 boost::mpl::quote_impl<boost::mpl::plus<mpl_::integral_c<int, 1>,
mpl_::int_<1>, mpl_::na, mpl_::na, mpl_::na>, true> (TemplateInstantiation)
#2 boost::mpl::quote5<plus, mpl_::void_>::apply<mpl_::integral_c<int,
1>, mpl_::int_<1>, mpl_::na, mpl_::na, mpl_::na> (TemplateInstantiation)
#3 boost::mpl::apply_wrap5<boost::mpl::quote5<plus, mpl_::void_>,
mpl_::integral_c<int, 1>, mpl_::int_<1>, mpl_::na, mpl_::na, mpl_::na>
(TemplateInstantiation)
...
#19 transform<vector_c<int, 1, 2, 3, 4, 5>, plus<_1, int_<1>>>::type



You can try Metashell from your browser: http://abel.web.elte.hu/shell
Or you can install it on your computer:
https://github.com/sabel83/metashell#version-200

Binary installers are available for a number of Linux distributions,
Windows and Mac OS X.

Feedback is welcome.

Regards,
Ábel


_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/
Benedek Thaler
2014-11-26 20:46:24 UTC
Permalink
Hi,

Great tool! Results of my second try below: (shame Firefox ignores
undescore, shift+insert works)
#msh evaluate std::is_pointer<char>
<stdin>:1:26: error: use of undeclared identifier 'std'
<stdin>:1:46: error: expected '(' for function-style cast or type
construction
<stdin>:1:48: error: expected unqualified-id


// You don't need the evaluate add pragma to evaluate this metaprogram. The
following command does this as
// well:
std::is_pointer<char>
std::is_pointer<char>
<stdin>:1:26: error: use of undeclared identifier 'std'
<stdin>:1:46: error: expected '(' for function-style cast or type
construction
<stdin>:1:48: error: expected unqualified-id

Probably I missed something, I just noted this because I would expect this
as the intuitive way.

Thanks,
Benedek

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Abel Sinkovics
2014-11-26 21:12:40 UTC
Permalink
Hi Benedek,
Post by Benedek Thaler
std::is_pointer<char>
<stdin>:1:26: error: use of undeclared identifier 'std'
<stdin>:1:46: error: expected '(' for function-style cast or type
construction
<stdin>:1:48: error: expected unqualified-id
Probably I missed something, I just noted this because I would expect this
as the intuitive way.
#include <type_traits>
std::is_pointer<char>::type
std::integral_constant<bool, false>

Regards,
Ábel


_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Mathias Gaunard
2014-11-29 17:56:36 UTC
Permalink
Post by Abel Sinkovics
Hi,
Metashell 2.0.0 has a template metaprogram debugger with a GDB-like
interface (thanks to András Kucsma), which can be useful for Boost
developers. It can show you how your metaprogram is executed
step-by-step. You can set breakpoints, step into or over the
metafunction calls and inspect the "template metaprogramming stack
trace" at any point during this: the chain of recursive template
#include <boost/mpl/transform.hpp>
#include <boost/mpl/vector_c.hpp>
#include <boost/mpl/plus.hpp>
#include <boost/mpl/lambda.hpp>
#include <boost/mpl/int.hpp>
using namespace boost::mpl;
#msh mdb transform<vector_c<int, 1, 2, 3, 4, 5>, plus<_1,
int_<1>>>::type
For help, type "help".
Metaprogram started
(mdb) rbreak ^boost::mpl::plus
Breakpoint "^boost::mpl::plus" will stop the execution on 25 locations
(mdb) continue
Breakpoint "^boost::mpl::plus" reached
boost::mpl::plus<mpl_::integral_c<int, 1>, mpl_::int_<1>, mpl_::na,
mpl_::na, mpl_::na> (Memoization)
(mdb) bt
#0 boost::mpl::plus<mpl_::integral_c<int, 1>, mpl_::int_<1>, mpl_::na,
mpl_::na, mpl_::na> (Memoization)
#1 boost::mpl::quote_impl<boost::mpl::plus<mpl_::integral_c<int, 1>,
mpl_::int_<1>, mpl_::na, mpl_::na, mpl_::na>, true> (TemplateInstantiation)
#2 boost::mpl::quote5<plus, mpl_::void_>::apply<mpl_::integral_c<int,
1>, mpl_::int_<1>, mpl_::na, mpl_::na, mpl_::na> (TemplateInstantiation)
#3 boost::mpl::apply_wrap5<boost::mpl::quote5<plus, mpl_::void_>,
mpl_::integral_c<int, 1>, mpl_::int_<1>, mpl_::na, mpl_::na, mpl_::na>
(TemplateInstantiation)
...
#19 transform<vector_c<int, 1, 2, 3, 4, 5>, plus<_1, int_<1>>>::type
From a glance, it appears to be missing important things to make it useful:

- pretty printing complex types so that you can actually read them
I don't think the formatter thing is a solution.
Consider an arbitrarily deep expression template tree, for example.

- easily get the source responsible for the instantiation
full path and line information for each element in the backtrace is
needed. Ideally you want to output that in a format where most smart
editors can directly recognize it and jump there in a click.

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cg
András Kucsma
2014-11-29 18:15:33 UTC
Permalink
Hi Mathias,

Thank you for the ideas!

Actually, both of those things are on our backlogs. Hopefully these will be
included in the next major release.

Regards,
András

On Sat, Nov 29, 2014 at 6:56 PM, Mathias Gaunard <
Post by Mathias Gaunard
Post by Abel Sinkovics
Hi,
Metashell 2.0.0 has a template metaprogram debugger with a GDB-like
interface (thanks to András Kucsma), which can be useful for Boost
developers. It can show you how your metaprogram is executed
step-by-step. You can set breakpoints, step into or over the
metafunction calls and inspect the "template metaprogramming stack
trace" at any point during this: the chain of recursive template
#include <boost/mpl/transform.hpp>
#include <boost/mpl/vector_c.hpp>
#include <boost/mpl/plus.hpp>
#include <boost/mpl/lambda.hpp>
#include <boost/mpl/int.hpp>
using namespace boost::mpl;
#msh mdb transform<vector_c<int, 1, 2, 3, 4, 5>, plus<_1,
int_<1>>>::type
For help, type "help".
Metaprogram started
(mdb) rbreak ^boost::mpl::plus
Breakpoint "^boost::mpl::plus" will stop the execution on 25 locations
(mdb) continue
Breakpoint "^boost::mpl::plus" reached
boost::mpl::plus<mpl_::integral_c<int, 1>, mpl_::int_<1>, mpl_::na,
mpl_::na, mpl_::na> (Memoization)
(mdb) bt
#0 boost::mpl::plus<mpl_::integral_c<int, 1>, mpl_::int_<1>, mpl_::na,
mpl_::na, mpl_::na> (Memoization)
#1 boost::mpl::quote_impl<boost::mpl::plus<mpl_::integral_c<int, 1>,
mpl_::int_<1>, mpl_::na, mpl_::na, mpl_::na>, true>
(TemplateInstantiation)
#2 boost::mpl::quote5<plus, mpl_::void_>::apply<mpl_::integral_c<int,
1>, mpl_::int_<1>, mpl_::na, mpl_::na, mpl_::na> (TemplateInstantiation)
#3 boost::mpl::apply_wrap5<boost::mpl::quote5<plus, mpl_::void_>,
mpl_::integral_c<int, 1>, mpl_::int_<1>, mpl_::na, mpl_::na, mpl_::na>
(TemplateInstantiation)
...
#19 transform<vector_c<int, 1, 2, 3, 4, 5>, plus<_1, int_<1>>>::type
- pretty printing complex types so that you can actually read them
I don't think the formatter thing is a solution.
Consider an arbitrarily deep expression template tree, for example.
- easily get the source responsible for the instantiation
full path and line information for each element in the backtrace is
needed. Ideally you want to output that in a format where most smart
editors can directly recognize it and jump there in a click.
_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/
mailman/listinfo.cgi/boost
_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.
Abel Sinkovics
2014-11-29 21:15:19 UTC
Permalink
Hi Mathias,

Thank you for checking it.
Post by Mathias Gaunard
- pretty printing complex types so that you can actually read them
I don't think the formatter thing is a solution.
Consider an arbitrarily deep expression template tree, for example.
We have some ideas on how to improve pretty printing of types and
displaying complex template instances based on our own experience.

How would you change/improve pretty printing in a way you find useful?
How would you display for example a deep expression template tree?

Regards,
Ábel


_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman
Mathias Gaunard
2014-11-29 22:06:55 UTC
Permalink
Post by András Kucsma
Hi Mathias,
Thank you for checking it.
Post by Mathias Gaunard
- pretty printing complex types so that you can actually read them
I don't think the formatter thing is a solution.
Consider an arbitrarily deep expression template tree, for example.
We have some ideas on how to improve pretty printing of types and
displaying complex template instances based on our own experience.
How would you change/improve pretty printing in a way you find useful?
How would you display for example a deep expression template tree?
It can be hard to match '<' and '>' together for long symbol names
involving instantiations of templates where the parameters are
themselves template instantiations.

I think it would be great to have an option to automatically indent the
type when printing so that it's easier to interpret.

Instead of
a<b,c<d>,e<f<g,h>>,i>, you could print it as

a<
b,
c<d>,
e<
f<g, h>
Post by András Kucsma
,
i
My example is still relatively short and uses one-letter names, but with
real cases it can quickly become unreadable (just try reading
Boost.Proto expression templates types without indentation, it's pretty
horrible)

In any case, my other suggestion is probably more important: I'd like to
be able to use this tool to track how a specific instantiation happens,
and since I'm working on a large project I'd need to have the reference
of where the template being instantiated is and where the instantiation
occurs.

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Larry Evans
2014-11-29 22:57:02 UTC
Permalink
Post by Mathias Gaunard
Post by András Kucsma
Hi Mathias,
Thank you for checking it.
Post by Mathias Gaunard
- pretty printing complex types so that you can actually read them
I don't think the formatter thing is a solution.
Consider an arbitrarily deep expression template tree, for example.
We have some ideas on how to improve pretty printing of types and
displaying complex template instances based on our own experience.
How would you change/improve pretty printing in a way you find useful?
How would you display for example a deep expression template tree?
It can be hard to match '<' and '>' together for long symbol names
involving instantiations of templates where the parameters are
themselves template instantiations.
I think it would be great to have an option to automatically indent the
type when printing so that it's easier to interpret.
Instead of
a<b,c<d>,e<f<g,h>>,i>, you could print it as
a<
b,
c<d>,
e<
f<g, h>
Post by András Kucsma
,
i
Or, keep *all* the delimiters(< , and >)
lined up, as in the following
( where the delimiters are lined up only
for the expr_prefx template):

expr_prefx
< op_ator< []>
, expr_prefx
< op_symb<lit,lit1>
, expr_prefx
< op_symb<act,action1>
This output was produced by:

template
< typename Operator
, typename... Operands
friend
std::ostream&
operator<<
( std::ostream& os
, expr_prefx
< Operator
, Operands...
)
{
os
<<"expr_prefx\n< "
<<indent_buf_in<<Operator{};
using swallow = int[]; // guaranties left to right order
(void)swallow
{ 0
, ( os
<<"\n"
<<indent_buf_out
<<", "
<<indent_buf_in<<Operands{}
, 0
)...
};
os<<indent_buf_out<<"\n>";
return os;
}

the indent_buf_in is an io manipulator that uses boost::iostreams
and an iostreams indent filter, which I can provide.

HTH.

-regards,
Larry



_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Abel Sinkovics
2014-11-30 07:37:48 UTC
Permalink
Hi Larry,
Post by Larry Evans
Or, keep *all* the delimiters(< , and >)
lined up, as in the following
( where the delimiters are lined up only
expr_prefx
< op_ator< []>
, expr_prefx
< op_symb<lit,lit1>
, expr_prefx
< op_symb<act,action1>
What we are planning to add is an interactive "template instance viewer"
which can recursively open/close template paramter lists similar to most
editor's support for showing/hiding blocks. It seems to me that this
feature could give the flexibility to concentrate on the important parts
of a complex template instance like this one.

Regards,
Ábel


_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Abel Sinkovics
2014-11-30 07:30:02 UTC
Permalink
Hi Mathias,
Post by Mathias Gaunard
It can be hard to match '<' and '>' together for long symbol names
involving instantiations of templates where the parameters are
themselves template instantiations.
I think it would be great to have an option to automatically indent
the type when printing so that it's easier to interpret.
Instead of
a<b,c<d>,e<f<g,h>>,i>, you could print it as
a<
b,
c<d>,
e<
f<g, h>
,
i
My example is still relatively short and uses one-letter names, but
with real cases it can quickly become unreadable (just try reading
Boost.Proto expression templates types without indentation, it's
pretty horrible)
Metashell has indentation support, but it is turned off by default since
it is not complete yet. You can experiment with it: you need to start it
with the --indent flag.
Post by Mathias Gaunard
In any case, my other suggestion is probably more important: I'd like
to be able to use this tool to track how a specific instantiation
happens, and since I'm working on a large project I'd need to have the
reference of where the template being instantiated is and where the
instantiation occurs.
Metashell uses Templight to get the template instantiation details and
to get the source location, Templight need to be able to provide that.
We are looking into it, so it will come.

Regards,
Ábel


_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Edward Diener
2014-11-30 19:08:29 UTC
Permalink
Post by András Kucsma
Hi Mathias,
Post by Mathias Gaunard
It can be hard to match '<' and '>' together for long symbol names
involving instantiations of templates where the parameters are
themselves template instantiations.
I think it would be great to have an option to automatically indent
the type when printing so that it's easier to interpret.
Instead of
a<b,c<d>,e<f<g,h>>,i>, you could print it as
a<
b,
c<d>,
e<
f<g, h>
,
i
My example is still relatively short and uses one-letter names, but
with real cases it can quickly become unreadable (just try reading
Boost.Proto expression templates types without indentation, it's
pretty horrible)
Metashell has indentation support, but it is turned off by default since
it is not complete yet. You can experiment with it: you need to start it
with the --indent flag.
Building Metashell on Windows seems pretty flawed:

1) Metashell depends on Templight.
2) Templight supposedly depends on an older version of Clang than what
is currently in the Clang svn source tree.
3) The instructions say "Download the source code from github" soon
followed by "Download the source code on your Windows host". Huh ?
4) The instructions tell me to build Clang with Templight. Suppose I
already have Clang regularly built from source using mingw/gcc. Am I
actually building another version of clang just for Metashell and using
that version instead as Clang when working with Metashell ? Ugh !!! It
would be much better if whatever is needed from Clang were part of the
latest clang source rather than some backdated version.
5) There is something about re-downloading the MingW header files. Is
his ever necessary ? Why should I be doing this on Linux if I am
building for Windows.
Post by András Kucsma
Post by Mathias Gaunard
In any case, my other suggestion is probably more important: I'd like
to be able to use this tool to track how a specific instantiation
happens, and since I'm working on a large project I'd need to have the
reference of where the template being instantiated is and where the
instantiation occurs.
Metashell uses Templight to get the template instantiation details and
to get the source location, Templight need to be able to provide that.
We are looking into it, so it will come.
_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Abel Sinkovics
2014-11-30 21:36:19 UTC
Permalink
Hi Edward,
Post by Edward Diener
1) Metashell depends on Templight.
2) Templight supposedly depends on an older version of Clang than what
is currently in the Clang svn source tree.
Metashell uses the functionality Templight provides. Since Templight is
not part of Clang (yet?), you need to build a Templight-enabled Clang
for Metashell. The patch is not following the Clang trunk continuously -
it is upgraded to the latest trunk sometimes. If you'd like to port the
patch to the latest in trunk, I'm happy to merge it.
Post by Edward Diener
3) The instructions say "Download the source code from github" soon
followed by "Download the source code on your Windows host". Huh ?
I've removed this extra instruction. Thank you for pointing it out.
Post by Edward Diener
4) The instructions tell me to build Clang with Templight. Suppose I
already have Clang regularly built from source using mingw/gcc. Am I
actually building another version of clang just for Metashell and
using that version instead as Clang when working with Metashell ? Ugh
!!! It would be much better if whatever is needed from Clang were part
of the latest clang source rather than some backdated version.
Of course it would be much easier if Templight was part of Clang, but it
isn't at the moment, so you need a special build. Let me know if you
have a better idea than this approach.
Post by Edward Diener
5) There is something about re-downloading the MingW header files. Is
his ever necessary ? Why should I be doing this on Linux if I am
building for Windows.
You probably don't need to do this, as they are part of the source tree.
The script it was done with is there, so you can see how they were
collected and I expect it to be useful for upgrading to newer versions
of these headers. The reason why it is a Linux script is that I found it
easier to write it this way - it shouldn't matter which platform you do
this on.

Regards,
Ábel


_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Edward Diener
2014-12-01 23:25:39 UTC
Permalink
Post by Abel Sinkovics
Hi Edward,
Post by Edward Diener
1) Metashell depends on Templight.
2) Templight supposedly depends on an older version of Clang than what
is currently in the Clang svn source tree.
Metashell uses the functionality Templight provides. Since Templight is
not part of Clang (yet?), you need to build a Templight-enabled Clang
for Metashell. The patch is not following the Clang trunk continuously -
it is upgraded to the latest trunk sometimes. If you'd like to port the
patch to the latest in trunk, I'm happy to merge it.
It would be more reasonable for the developers of Templight to port
their patch to the latest clang trunk. I e-mailed them suggesting this,
but did not receive a reply as yet.
Post by Abel Sinkovics
Post by Edward Diener
3) The instructions say "Download the source code from github" soon
followed by "Download the source code on your Windows host". Huh ?
I've removed this extra instruction. Thank you for pointing it out.
Post by Edward Diener
4) The instructions tell me to build Clang with Templight. Suppose I
already have Clang regularly built from source using mingw/gcc. Am I
actually building another version of clang just for Metashell and
using that version instead as Clang when working with Metashell ? Ugh
!!! It would be much better if whatever is needed from Clang were part
of the latest clang source rather than some backdated version.
Of course it would be much easier if Templight was part of Clang, but it
isn't at the moment, so you need a special build. Let me know if you
have a better idea than this approach.
Post by Edward Diener
5) There is something about re-downloading the MingW header files. Is
his ever necessary ? Why should I be doing this on Linux if I am
building for Windows.
You probably don't need to do this, as they are part of the source tree.
The script it was done with is there, so you can see how they were
collected and I expect it to be useful for upgrading to newer versions
of these headers. The reason why it is a Linux script is that I found it
easier to write it this way - it shouldn't matter which platform you do
this on.
I was able to build Metashell successfully under Windows with your
instructions. Two further suggestions for your "Building Metashell on
Windows" instructions:

1) The Visual Studio build for Metashell should I believe specify that
the configuration should match that of Clang build, ie. 'release'. Or do
I have that wrong and Metashell is supposed to be built/run as the
'debug' version ?

2) You should specify where the final metaclass.exe will be built.
Evidently it is in the bin/app/Release directory for me.

I will try out your product. It is great to see you working on this as
it may be a great boon to template metaprogramming.


_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Abel Sinkovics
2014-12-02 21:41:41 UTC
Permalink
Hi Edward,
Post by Edward Diener
I was able to build Metashell successfully under Windows with your
instructions. Two further suggestions for your "Building Metashell on
1) The Visual Studio build for Metashell should I believe specify that
the configuration should match that of Clang build, ie. 'release'. Or
do I have that wrong and Metashell is supposed to be built/run as the
'debug' version ?
I've updated the instructions. You can do either a Debug or a Release
build against the Release Clang build.
Post by Edward Diener
2) You should specify where the final metaclass.exe will be built.
Evidently it is in the bin/app/Release directory for me.
I've added this as well.

Thank you for the suggestions.

Regards,
Ábel



_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Mikael Persson
2014-12-02 23:25:16 UTC
Permalink
Hi all,

I've been working on a re-vamped version of Templight (
https://github.com/mikael-s-persson/templight). So, I have a few things to
mention in relation to the points raised in this discussion that relate to
issues with the templight patch that metashell relies on.

First thing's first, my version of templight also includes a GDB-style
template meta-program debugger:

https://github.com/mikael-s-persson/templight#using-the-templight-debugger

The crucial difference that it runs on-line... i.e., breakpoints will pause
the compilation to provide you with a shell that allows you to query
information from the AST directly ("while it's hot"). It is not complete or
thoroughly tested yet, but it's pretty powerful (e.g., allows you to, for
example, evaluate the value of constants expressions in the context of the
template instantiation or to evaluate the actual type of typedefs and
similar dependent types). Technically, the possibilities are unlimited as
far as gaining information about whatever you want during the compilation
(suggestions are welcome!).

The point was raised that a problem with the original templight patch is
that it requires people to sync to a much older version of Clang, and
support for Windows is not great. My version of templight solves those
problems by having a much more up-to-date patch for Clang, that should work
on the latest svn trunk of it. Also, I generate separate applications for
Templight ("templight(.exe)", "templight++(.exe)" and "templight-cl.exe")
which means that you don't have to deal with a templight-enabled clang
executables that might interfere with your other workflows. And evidently,
with templight-cl.exe, you have a MSVC-compatible version of templight too,
based on the latest work on clang-cl.exe as a drop-in replacement for
cl.exe. The patch on Clang is very minimal (part of my re-vamping was to
pull most of the templight code out of the main clang code-base, and only
leaving the minimal hooks), that patch has been submitted to clang, but it
is still pending approval (I probably need more pressure from the community
to get this patch through).
Post by Mathias Gaunard
pretty printing complex types so that you can actually read them
...
It can be hard to match '<' and '>' together for long symbol names
involving instantiations of templates where the parameters are themselves
template instantiations.

The basics of this has been implemented in my version of Templight. I
created a protobuf-based output format in which template names are
compressed by re-using the repeating sub-strings in the names. To achieve
this, I had to use the natural splits that exist in template names, based
on matching "<", ",", and ">" delimiters to extract template arguments from
the complex template instantiation names. I've tested this quite
extensively, on many of the example programs from Boost libraries like
Spirit and Proto, and it works quite well (I'm sure there are some
corner-cases where it breaks down, but I haven't encountered any so far).
When you output a protobuf-formatted template instantiation trace from
templight, you actually get a linked-tree type of structure which you could
easily use to make any kind of pretty-printing you like (or fancy
collapse-able stuff too).

Abel, I recommend that you check out that implementation, as it's really
simple yet effective (also, the compression ratio and processing speed
difference compared to the original YAML outputs is huge):

https://github.com/mikael-s-persson/templight/blob/master/templight_messages.proto
https://github.com/mikael-s-persson/templight/blob/master/lib/TemplightProtobufWriter.cpp
https://github.com/mikael-s-persson/templight/blob/master/lib/TemplightProtobufReader.cpp
Post by Mathias Gaunard
full path and line information for each element in the backtrace is
needed. Ideally you want to output that in a format where most smart
editors can directly recognize it and jump there in a click.

Of course, this is necessary! That is what I have done on my templight
debugger. I don't know why Abel's MDB wouldn't have that. The whole point
of a GDB-style debugger is to run it behind a smart editor in the fashion
you describe.
Post by Mathias Gaunard
I'd like to be able to use this tool to track how a specific
instantiation happens, and since I'm working on a large project I'd need to
have the reference of where the template being instantiated is and where
the instantiation occurs.

Currently, my templight debugger does not have a command to query the
location of where the template being instantiated is, only where the
instantiation occurs. But that is only for historical reason (this is what
templight traces record), but I could easily add this feature. I'll work on
it.
Building anything on Windows is flawed ;)

For the record, my version of templight, that includes the interactive
debugger builds fine under Windows, and has the MSVC-compatible
templight-cl.exe program too.
Post by Mathias Gaunard
2) Templight supposedly depends on an older version of Clang than what is
currently in the Clang svn source tree.

My version is up-to-date and much less intrusive on Clang, and therefore,
with fewer foreseeable problems in applying it to whatever version of the
Clang source tree you are sync'd with (as long as it is not too old).
However, my version of templight is not currently compatible with metashell
/ MDB, because that's not my responsibility (but it is my hope).
Post by Mathias Gaunard
Suppose I already have Clang regularly built from source using mingw/gcc.
Am I actually building another version of clang just for Metashell and
using that version instead as Clang when working with Metashell ? Ugh !!!
It would be much better if whatever is needed from Clang were part of the
latest clang source rather than some backdated version.

That is a point I have already raised with Abel some time ago. Clearly,
this is a very impractical situation to be in. This was the whole reason
for minimizing the patch necessary on Clang and then submitting it for
permanent inclusion in Clang. I have done this, just waiting for the Clang
gate-keepers to manifest any kind of interest.


I just wanted to point these things out. I'm very enthusiastic about the
future of this, and I believe it will have to go through my version of
Templight, to have any hope of success in the near future.

Cheers,
Mikael Persson.

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Edward Diener
2014-12-03 13:11:07 UTC
Permalink
Post by Mikael Persson
Hi all,
I've been working on a re-vamped version of Templight (
https://github.com/mikael-s-persson/templight). So, I have a few things to
mention in relation to the points raised in this discussion that relate to
issues with the templight patch that metashell relies on.
First thing's first, my version of templight also includes a GDB-style
https://github.com/mikael-s-persson/templight#using-the-templight-debugger
The crucial difference that it runs on-line... i.e., breakpoints will pause
the compilation to provide you with a shell that allows you to query
information from the AST directly ("while it's hot"). It is not complete or
thoroughly tested yet, but it's pretty powerful (e.g., allows you to, for
example, evaluate the value of constants expressions in the context of the
template instantiation or to evaluate the actual type of typedefs and
similar dependent types). Technically, the possibilities are unlimited as
far as gaining information about whatever you want during the compilation
(suggestions are welcome!).
The point was raised that a problem with the original templight patch is
that it requires people to sync to a much older version of Clang, and
support for Windows is not great. My version of templight solves those
problems by having a much more up-to-date patch for Clang, that should work
on the latest svn trunk of it. Also, I generate separate applications for
Templight ("templight(.exe)", "templight++(.exe)" and "templight-cl.exe")
which means that you don't have to deal with a templight-enabled clang
executables that might interfere with your other workflows. And evidently,
with templight-cl.exe, you have a MSVC-compatible version of templight too,
based on the latest work on clang-cl.exe as a drop-in replacement for
cl.exe. The patch on Clang is very minimal (part of my re-vamping was to
pull most of the templight code out of the main clang code-base, and only
leaving the minimal hooks), that patch has been submitted to clang, but it
is still pending approval (I probably need more pressure from the community
to get this patch through).
Post by Mathias Gaunard
pretty printing complex types so that you can actually read them
...
It can be hard to match '<' and '>' together for long symbol names
involving instantiations of templates where the parameters are themselves
template instantiations.
The basics of this has been implemented in my version of Templight. I
created a protobuf-based output format in which template names are
compressed by re-using the repeating sub-strings in the names. To achieve
this, I had to use the natural splits that exist in template names, based
on matching "<", ",", and ">" delimiters to extract template arguments from
the complex template instantiation names. I've tested this quite
extensively, on many of the example programs from Boost libraries like
Spirit and Proto, and it works quite well (I'm sure there are some
corner-cases where it breaks down, but I haven't encountered any so far).
When you output a protobuf-formatted template instantiation trace from
templight, you actually get a linked-tree type of structure which you could
easily use to make any kind of pretty-printing you like (or fancy
collapse-able stuff too).
Abel, I recommend that you check out that implementation, as it's really
simple yet effective (also, the compression ratio and processing speed
https://github.com/mikael-s-persson/templight/blob/master/templight_messages.proto
https://github.com/mikael-s-persson/templight/blob/master/lib/TemplightProtobufWriter.cpp
https://github.com/mikael-s-persson/templight/blob/master/lib/TemplightProtobufReader.cpp
Post by Mathias Gaunard
full path and line information for each element in the backtrace is
needed. Ideally you want to output that in a format where most smart
editors can directly recognize it and jump there in a click.
Of course, this is necessary! That is what I have done on my templight
debugger. I don't know why Abel's MDB wouldn't have that. The whole point
of a GDB-style debugger is to run it behind a smart editor in the fashion
you describe.
Post by Mathias Gaunard
I'd like to be able to use this tool to track how a specific
instantiation happens, and since I'm working on a large project I'd need to
have the reference of where the template being instantiated is and where
the instantiation occurs.
Currently, my templight debugger does not have a command to query the
location of where the template being instantiated is, only where the
instantiation occurs. But that is only for historical reason (this is what
templight traces record), but I could easily add this feature. I'll work on
it.
Building anything on Windows is flawed ;)
For the record, my version of templight, that includes the interactive
debugger builds fine under Windows, and has the MSVC-compatible
templight-cl.exe program too.
Post by Mathias Gaunard
2) Templight supposedly depends on an older version of Clang than what is
currently in the Clang svn source tree.
My version is up-to-date and much less intrusive on Clang, and therefore,
with fewer foreseeable problems in applying it to whatever version of the
Clang source tree you are sync'd with (as long as it is not too old).
However, my version of templight is not currently compatible with metashell
/ MDB, because that's not my responsibility (but it is my hope).
Post by Mathias Gaunard
Suppose I already have Clang regularly built from source using mingw/gcc.
Am I actually building another version of clang just for Metashell and
using that version instead as Clang when working with Metashell ? Ugh !!!
It would be much better if whatever is needed from Clang were part of the
latest clang source rather than some backdated version.
That is a point I have already raised with Abel some time ago. Clearly,
this is a very impractical situation to be in. This was the whole reason
for minimizing the patch necessary on Clang and then submitting it for
permanent inclusion in Clang. I have done this, just waiting for the Clang
gate-keepers to manifest any kind of interest.
I just wanted to point these things out. I'm very enthusiastic about the
future of this, and I believe it will have to go through my version of
Templight, to have any hope of success in the near future.
I think this is great. I surely hope that templight can become part of
the clang build-from-source process, as well as having the clang
compiler switches for templight documented. The ability to unroll
template instantiations by the compiler, so that the template
metaprogramming can see what is happening at each step when a template
is instantiated, is a great boon for the template metaprogrammer.



_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Continue reading on narkive:
Loading...