How does the current LLM hype affect us?

It’s been a while since I was here, but I haven’t disappeared - no worries.

The recent hype around ChatGPT drew me to this forum again to verify some thoughts.

OpenAI unleashed a beast by publishing ChatGPT (and shortly also ChatGPT Pro), with GPT-3 as a commercial product in the background. I did a. little bit of investigation, just one evening into how they offer these things and how ChatGPT responds to certian things. That made me wonder…

What if we can use the training/filter interfadce of GPT-3 to define a DSL, by introducing concepts and relations - and then feed the tool requests expressed in this DSL, would be be getting closer to an NL based language workbench with integrated code generation? Has anyone looked into this already, and maybe has some experiences to share?
As said in one evenining I didn’t get passed the idea stage, but I am curious to see and try more - if more people are intersted. Doing research on your own is no fun.

2 Likes

I think @jcabot has spent plenty of thought energy on this. It’s certainly interesting. Maybe I’m subconsciously a bit hesitant to investigate this angle myself because it’d might me partially redundant.

1 Like

I think once the API for ChatGPT is available and can be used for a DSL-driven fine-tuning, we’ll be able to teach chatGPT some DSL concepts and use it for a NL-to-ourDSL transformation.

Right now, ChatGPT knows at least plantUML syntax (and OCL!) so you can explore how good it is at expressing UML models and OCL constraints from a textual specification.

To me, it will become one more tool in our toolbox.

I do see it even more useful for code-generation as the input and the output have both more clear semantics (the grammar of our DSL and the grammar and the target programming language). I wrote down some thoughts on this here: https://modeling-languages.com/models-to-code-models-to-prompts/

1 Like

You refer to the chatGPT API. Will that be much different from the GPT-3 API - which in my view might be able to do a large part of what you say already. Have you looked at that? (not read your article, so my apologies if it’s addressed there)

I think so. GPT 3.5 (on which chatgpt is based) seems to know much more. I did some experiments with OCL with GPT3 and the results were very bad (most times the OCL was incomplete or plain wrong, only SQL queries seem to be mastered) while the same type of expressions are well managed by GPT3.5 (so you can ask chatgpt for a much larger set of NL - to - OCL translations)

Just sharing some related articles I found interesting:

@voelter: Will AI replace Models and DSLs?

@andreas.muelder AI-powered no-code platforms — how to use chatGPT to abstract domain languages