How to test a DSL?

Hi all,

I am developing a DSL and wanted to write some tests for the parser, but I am not sure how many or what is “enough” to test. ATM I have a couple of tests that parse small snippets and just check that there are no parse errors. I am starting to write another set of tests that parse a file and then finds specific instances in the AST and checks that the properties are being correctly set according to the CS.

Is this a good approach? And more importantly how complete should the tests be? Should I have a test file that exercises all the language constructs?

Any pointers are welcome.

“It depends…” is the only right answer.

What you’re already doing now sounds good for testing the parsing part of a DSL. There are more aspects to a DSL though, such as: constraints (with underlying type system, possibly), generation/interpretation (“semantics”), editor support (content assist), etc.

In general, a couple of well-crafted “end-to-end”-like tests work wonders to be alerted to any problem introduced. Especially when generating code from DSL content, there’s a “fan out”-effect so that any problem with the surface area of the DSL (the parser, in your case) will be almost guaranteed to show itself in some form in the generated code.