Are code generation "metaprogramming" tools a well-developed thing?

[u/math_code_nerd5]

Now lots of people are talking about LLMs for coding to speed up the repetitive parts of coding, reduce the time spent on boilerplate, etc. But this makes it sound as though writing and manipulating code programmatically/algorithmically wasn't a thing before LLMs. However, although I didn't hear much about them, looking back it's surprising if people never thought about building something like this.

Of course, all compilers generate code in a lower-level form from higher-level code, however most of these are not interactive, i.e. they don't generate code in human-readable form, don't explain their steps to the user, and are not intended to generate a first pass of code for a programmer to then take and modify. What I'm imagining is a large category of tools, including anywhere from ones that auto-generate things like bounds checks and testing variables for null before using them, to drag-and-drop graphical tools a la Scratch that effectively allow you to drag loops into place around your code and have a loop condition and brackets appear, to any number of scriptable recipes where you can effectively say "take this block I've written for x1, and repeat the same pattern for y1, x2, y2, etc" and you won't have to type each one out individually. This could potentially allow a lot of the productivity gains and saved attention spent on details that people seem to be getting from LLMs without the cost, resource use, and philosophical implications, which is something I'd definitely welcome.

Of course you could write a code generator from scratch in something like Python just using regular string handling, but I'm wondering if there are already plugins like this for many IDEs that I just don't know about because they're not talked about much. Since so much of programming is about recursive abstractions, the idea that programmers would not have tried the idea of tools that you can code to spit out more code would strike me as very odd. And doing a bit of research, I've come across things like this: https://github.com/imatix/gsl#a-short-history-of-code-generation and this: https://tifoputra.medium.com/building-your-own-swift-code-generator-using-swift-script-78470eef0206 that suggest there is substantial work already toward making something like this.

[u/drBearhands]

Yes, this was always possible, but neither businesses nor most programmers ever cared. If you wanted to actually solve the problems LLMs are prophetized to solve (in SE) you would not use an LLM.

[u/Inevitable-Ant1725]

Generally languages with higher level constructs aren't particularly composable from simpler languages.

Take, for instance, the features of dynamically typed variables, garbage collection and continuations and imagine the horrible, unreadable mess that Chicken Scheme generates when it uses the Cheney on the MTA and continuation passing to turn Scheme into C.

The advanced constructs aren't a slight expansion of code, they're complex, deeply integrated runtimes and compiler constraints.

[u/initial-algebra → u/Inevitable-Ant1725]

I don't think there's any fundamental reason that global features like that couldn't be implemented with metaprogramming. I mean, a compiler is a metaprogram, no? The real question is, what are macros missing that compilers have? I think the answer lies in the literal meaning of the word "compiler", or someone that compiles information together from multiple sources. (Ironically, programmers decided at some point to call that linking, with direct source-to-machine code translation being compiling.) Normally, macros are purely local, with limited to no ability to communicate, but what if the compiler's ability to, well, compile multiple definitions together could be harnessed? If you can write a macro that operates on an entire (sub)program as a graph data structure, you can add a new layer to, or even outright replace the compiler backend (or have multiple backends for multitier programming, especially when some of the target code is data for another target, e.g. the client-side JavaScript served by a Web application).

[u/Inevitable-Ant1725 → u/initial-algebra]

I'm planning on trying to write a compiler compiler system like LLVM, but the IR that you generate will be a high level language with a hell of a lot of types and options instead of a low level language.

And what it gets turned into to be optimized is graph, not a low level language.

I imagine there might be a medium level representation graph that's more like an IR and then finally machine instructions in a graph that maintains all dependencies and a lot of extra arcs.

So code doesn't get represented by simpler languages it gets represented by more and more information dense graphs, probably connected to higher representations so that deoptimization in a jit is instant.

[u/kwan_e → u/Inevitable-Ant1725]

I'm pretty sure the big three compilers do this already. They have many different graph representations for different optimization strategies.

[u/initial-algebra]

What I'm imagining is a large category of tools, including anywhere from ones that auto-generate things like bounds checks and testing variables for null before using them, to drag-and-drop graphical tools a la Scratch that effectively allow you to drag loops into place around your code and have a loop condition and brackets appear, to any number of scriptable recipes where you can effectively say "take this block I've written for x1, and repeat the same pattern for y1, x2, y2, etc" and you won't have to type each one out individually.

I'm not sure I would really class this as metaprogramming, it sounds more like editor features like snippets, autocomplete etc. LLMs are just really powerful autocomplete, after all.

Metaprogramming leaves the original source code intact, and generally the output is not meant to be human-readable, nor is it necessarily even textual. It has always been common practice to use macros for things like what you've mentioned, at least until more sophisticated type checking and higher-order functions replaced them in many cases (I mean, bounds checking and null testing? Those have been solved a long time ago.)

I think the reality is that many people are stuck writing in crappy programming languages at work, and between not caring enough to learn the "proper" way to reduce keystrokes and corporate forcing AI on them anyway, they just brute force it. This is a social problem, not a technical problem.

[u/kwan_e → u/initial-algebra]

This is a social problem, not a technical problem.

I absolutely agree with this. LLMs being able to generate a lot of code has really depressed the desire to clean up designs to be easier to write. It's just papering over technical debt.

[u/Inconstant_Moo]

I think where there aren't already IDE tools for a thing that's usually because it lies at the intersection of hard to do, hard to use, and not very useful.

Take bounds checks, for example. Suppose I'm programming in Go, as I do, and I have a expression y := foo([bar(x)-1]. I highlight it and a guard if bar(x)-1 < 0 || bar(x)-1 >= len(foo) {y := foo([bar(x)-1]} else {} appears. Now, the question of what to put in the else block and what I can possibly do about it is a puzzler because how does one handle a bounds error? The repetition of bar(x)-1 is merely annoying. But the really sucky thing about that is that now y is only in scope in the first branch of the if block so my code no longer compiles.

So what ought the IDE to do, and when will I want to use it for what?

"Drag loops into place around your code": or I could write the loops and then write code inside them, and the great thing about doing that is that the IDE will know the names and types of the loop variables when I'm writing the body of the loop.

There are Java boilerplate generators in IntelliJ for things like (IRRC) constructors and comparisons 'cos it's so verbose that that's actually useful. But it does really have to be the dullest sort of boilerplate. Trying to generate your flow of control is going to be more trouble than it's worth.

[u/yjlom → u/Inconstant_Moo]

Snippet engines like yas are pretty popular for good reason.

For example, when I write C, I have r <tab> bound to (more or less): Prompt for $1, default i; prompt for $2, default len; insert for(unsigned $1 = $2; $1 --> 0;){\n\s\s$0\n}, indented; place cursor at $0.

It saves a fair bit of time while being more readable and polluting namespaces less than macros.

[u/Inconstant_Moo → u/yjlom]

Snippet engines like yas are pretty popular for good reason.

And so that is a thing that exists, unlike dragging a loop around the body after you've written it, which no-one wants. (Indeed, your short-cut with the loops is so useful and commonplace and easy to implement that in modern languages it's been turned into a range-style loop. )

My point wasn't that this sort of short-cut is always bad, but that we've probably already found nearly all the good ones by now.

There are remarkably few ideas that are useful and easy to think of and possible to do which haven't been done yet. One of the problems with OPs idea is that he hasn't really thought of his idea, he's just vaguely gesturing towards it. OP would find it more challenging to think: "Suppose I was programming in language A and I wanted to do thing B and I had written code C. Then I would like to be able to highlight subsection D, press Ctrl+E, and have thing F happen."

And if OP comes up with something we don't already have, then they can write an IDE extension and we'll have it.

While OP's not being concrete, they may well be just gesturing at a nonexistent space between the things we have and the things we want. A lot of "ideas" about software are like this.

[u/tending]

They operate at very different levels. Lisp macros are great for being able to present an interface to library users that lets them directly express what they're thinking about while still giving the library author tons of implementation flexibility, but they still generate code in a mechanical way and only work on the input forms the macro author specifically anticipated. The strength of LLMs is generating code from fuzzy descriptions.

[u/ShadowPages]

Frankly, I've never seen an implementation that could generate useful code without requiring you to specify the required functionality in so much detail that even a novice developer could write the code with less information.

[u/jason-reddit-public]

C macros are primitive code generators especially with the "X macros" idiom. I'm sure some projects use m4, Python, etc. to generate code. Google protocol buffers and things like that are a pretty common way code generation finds its way into some applications.

[u/RecursiveServitor]

Roslyn Source Generators may be of interest. They emit code into the compilation based on user code. The neat part is that user code can reference the generated code without any setup. It just works.

[u/CastleHoney]

I've seen this done at a framework level, namely in Ruby on Rails and ASP.NET (C#). Rails comes with CLI commands for generating boilerplate-y code such as controllers (in the MVC model sense) and DB migrations. It even allows users to write their own generators. ASP.NET, when used with Visual Studio at least, offers something similar, where the user fills out a form and the VS will automatically generate boilerplate according to the form. I think this sort of functionality doesn't belong in a language, but rather in frameworks. A language is too general-purpose.

[u/dnpetrov]

The basic code generation scenario for LLMs is not to automate repetitive tasks. LLMs can do that, but they are surprisingly not that good at it; see InstrEditBench, for example. Rather, LLMs are used to generate code in a programming language, given a description (specification, if you will) in a natural language.

[u/schakalsynthetc]

The work certainly is out there and most of the industry is already using it, in some form. Part of the problem is that we keep quietly redefining "programming" so that it's all baked into our notion of just what "programming" is, instead of being widely noticed and understood.

I mean, ultimately, we write code in higher-level programming languages for the same reason mathematicians use hand-written mathematical notation instead of stating everything in natural language: Reading and writing notation is a skill that takes time and effort to acquire and makes things look intimidating to newcomers, but it's worth it because there's nothing in natural language that quite matches the clarity and compactness you can achieve with it.

Ideally, a good high-level programming language is exactly this kind of thing. If you're writing a CS paper, you'll naturally find yourself including code snippets here and there because you'll have things you want to say for which the code is the clearest, most effective way to say it. I don't see that changing much just because the papers are now being read by AI agents as well as humans.

So, in some ways these things (LLMs vs metaprogramming, high-level language design and smart compilers), are tools to the same ends by different strategies, and neither one is wholly up to the job of supplanting the other.

Terry Tao, for example, uses a mix of LLMs and Lean, which is a proof assistant in the lineage of Coq et al and developed alongside typed functional programming languages. But the general public only see the LLM because don't really consider Lean as an integral part of whatever they're referring to when they enthuse about "AI" in this context. Which is unfortunate because it really is, and they really should.

[u/church-rosser]

ANSI Common Lisp has been Metaprogrammig since at least the early 1990s

[u/math_code_nerd5]

Per AutoModerator's request I hereby confirm that this project did not use an LLM as part of the development process. Its last commit was 9 years ago and thus pre-dates LLMs.

[u/catbrane]

Sure, it's usually called refactoring, try googling for the tools. There are loads of plugs and add-ons for IDEs like Visual Studio to do this.