why extern template?

[germasch]

As I've been dealing with lots of ADIOS2_FOREACH_* uses, I've come to wonder why extern template is used (pretty frequently).

Example from bindings/CXX11/cxx11/Types.h

template <class T>
std::string GetType() noexcept;

// LIMIT TEMPLATE TYPES FOR adios2::GetType
#define declare_template_instantiation(T)                                      \
                                                                               \
    extern template std::string GetType<T>() noexcept;

ADIOS2_FOREACH_TYPE_1ARG(declare_template_instantiation)
#undef declare_template_instantiation

My understanding is that extern template is used to prevent the compile from creating instantiations of the given template on use (ie., in later in the file which includes this header). That makes sense in that the necessary instantiations are explicitly generated in Types.cpp, so one doesn't need them elsewhere, and, if this was a long function at least, one could save compile time and disk space during the build.

However, since adios puts template definitions into .tcc files in the case where they are explicitly instantiated, there is no way that the template could be instantiated elsewhere in the first place. So considering how you prescribe to organize the source files, the extern template seem to be redundant?

[williamfgc]

@germasch definitions in .tcc files are not visible to the consumer of .h , as they are not really inline functions. In practical terms .tcc acts as a natural extension to .cpp, which always includes .h and .tcc. Your first paragraph applies to tcc as well as cpp files, in which extern and the MACRO in .h will provide the scope of the supported types. And your conclusion is correct, for the developer that re-compiles the entire code several times a day it would be a headache if templates were inlined (in .h or .inl) as compilation times will go to the roof.

[germasch]

I'm not suggesting to change anything about the .h / .inl / .tcc / .cpp structure -- that works well. My point is that I believe the extern template in the header file doesn't actually do anything:

• When the header file is include from the corresponding .cpp file, it tells the compiler to not instantiate those templates -- but then they get defined in the .tcc and explicitly instantiated in the .cpp, so they do get instantiated despite the extern template, so the extern template doesn't actually have any effect.
• When the header file is included somewhere else, the extern template tells the compiler to not instantiate those templates (when used), but they couldn't possibly be instantiated anyway in this case, since they're not even defined (because they're defined in the .tcc which isn't included in this case). So the extern template again doesn't have any effect.

So the only effect that extern template has, in my opinion, is to remind humans reading the header file that those templates are instantiated elsewhere.

[williamfgc]

So the only effect that extern template has, in my opinion, is to remind humans reading the header file that those templates are instantiated elsewhere.

Not just remind humans, but tells the compiler explicitly to move the search for the definition (thus preventing several inlining or errors if they are not found) to the linking stage. It disables implicit instantiation: https://isocpp.org/wiki/faq/cpp11-language-templates#extern-templates extern in templates is something introduced in C++11.

[germasch]

I agree with all you say, but I think I still didn't get my point across. So I actually did an experiment:
I removed all the extern template from core/*.h, and compared the size of the resulting library as well as some object files.

• The resulting libadios.so is exactly the same size. That's actually expected no matter what, so not very useful, as in the final link, definitions of templates instantiated more than once will be dropped (but one).
• The resulting object files are also exactly the same length -- I looked at bindings/CXX11 as an example. If the extern template was really preventing any instantiations, one would see smaller object files in the case of extern template having its intended effect.

The reason that extern template does not have any effect is that it prevents implicit template instantiations of the specified functions/classes. However, implicit template instantiation can only possibly ever happen if the definition of that function is known (ie., part of the compilation unit). Since the definition is in the .tcc files, which are not included, the definitions are not known, and the compiler cannot possibly try to generate code for a function that it has only seen been declared, but not defined.

extern template would have uses for templates which are defined in .inl files, but it doesn't do anything for templates which are only defined in .tcc.

[williamfgc]

@germasch having extern does have an effect on tcc definitions, having it in h triggers the error: specialization of ...after instantiation compiler error.

[germasch]

@williamfgc: You mean just removing the extern from the statements in the .h triggers that error? That would make it requests to instantiate those templates, which is bad a number of reasons:

• if there is a (general) definition, the (generic) template will be instantiated for the listed types. If it is then specialized later, the error you're getting would make sense (essentially, the wrong version has already been instantiated.)
• though often, no general version of the template is even defined (just declared). in that case you'll get an error that the function can't be instantiated without being defined first.
• if it wasn't for those errors, the compiler would instantiate all those templates every time the header file is included, which would be horrible for compile / link performance.

I may have misunderstood you in the first place, but anyway, I'm not suggesting to just remove the extern but the entire code blocks doing the extern template for functions which are defined in .tcc files. (Functions defined in .inl are possibly a different story, but I'm not sure extern template is even currently used on any such functions.)

[williamfgc]

@germasch no, I meant the whole extern function definition, as you in inferred the definition without extern is pointless. In any case, to me is like the use of const, if you are not changing a variable then it looks like it's useless (or doing anything) in existing code...but it's still a compiler hint and improves the communication among developers. Also, after a make install (e.g. modules, binary distributions) it's the only way users can know what types are accepted (ADIOSMacro.h is public) as tcc files are private.

inl files in bindings/CXX11 are public, and there is only one (ADIOS.inl), as they are always included at the bottom of the header, extern wouldn't have any effect as they are visible to anyone including ADIOS.h . Anyways, template separation is a good thing, keeps developers aligned.

[germasch]

Alright, so I guess we are in agreement that it's essentially a human-only thing, then. So I'll close this issue, as that's you guys' coding style decision.