Thoughts on visual programming

I will be the first to concede that visual programming brings some interesting opportunities, mainly to democratize software development, even if only as a lure.

However, I see visual vs textual as an less important dichotomy. The key aspect is really the raise in the level of abstraction required so that users can create large, professional applications without having to specify every little detail by hand. That is a requirement for visual programming to work, as it relies on a media that does not scale to a finer level of detail. But raising the level of abstraction is a very beneficial feature for text-based programming as well.

If we separate a language from its concrete notation (like UML does, and a bit like the JVM/CLI promote for JVM/CLR-based languages, but to a lesser extent), what is left is is abstract syntax and semantics (95% of the language, if you ask me). Whatever abstract syntax and semantics could lead us to a successful visual programming language, could lead us to very interesting text-based languages as well.

I am not saying visual programming does not face some particular challenges on its own. The article covers some of that. But the key one, IMO, is about raising the level of abstraction, and that is a challenge that when solved can benefit all forms of programming, not only visually or graphically oriented ones. So let’s not get distracted by the visual vs. text vs. projectional flame wars. We are all on the same boat, and we are fighting the same enemy: the insufficient level of abstraction for software applications provided by the mainstream tools.

1 Like

Yes, I think many non-developers see a wall of text and are scared, while drag-and-drop seems easy, so it is great to lure them in until they are more productive with text

Yes, but is much more difficult to communicate

From my point of view raising the level of abstraction is so much easier if we focus on specific domains. For doing so we need to lure in domain experts and convince their employers to pay for the benefit. If visual programming helps in that direction I see it as a mean to an end

1 Like

Drakon aka Dragon in English language was created through soviet/russian space program.


http://drakon-editor.sourceforge.net/

1 Like

After reading above post, though I do not know about DSLs much apart from what I am reading in this forum and still studying (I wanted to ask guidance on books, will post), sharing the thoughts as they come. Please feel free to correct as I share my views, which if anybody classify as fiction, I do agree but only correct them saying crazy is probably the right word though I see no possibility to at least not to achieve with current technology requiring the effort which might be falling anywhere between relatively tremendous to a risk of unachievable when mathematically modeled for a real working one. Now, here is some arrangement for the visual DSL programming:

  1. input:

*. Developers are also comfortable with ‘already known’: Eg: IA64 Vs x86_64

*. Lot of objects around in real world for people to build and convey ideas: Right from simple lego type to complex research level details can be found in them and some of which are already evolved or existing to be used as is for the domain. The single most important one is the concept of 3d space over a time line.

*. Any idea or construct which is not in real world, like a time machine but needed to convey the idea, may be offered as part of real world object (Hey, we do have a time machine, in an artistic way, at least in lot of science fiction movies and we can borrow just like from the movie, last action hero) offered by the input part.

  1. interpret:

*. Principle of mathematical induction: prove for 1,n,n+1: proved for all. Only this we have to teach the users to understand in conveying the idea and train them.

*. The goal seek-ed is built by the customer and whatever orthogonal-ly (she/he thinks) different ideas which can be conveyed as part of it are arranged and finalized. For example, searching is done by a voice call to the object and the object is located. This is conveyed by building a model with the interface given. Similarly the sorting and any other transformation or task.

  1. When the above boundaries are fixed, any graphics DSL belonging to this class is now defined by,

*. The predefined mappings (limited to virtually infinite) of these objects (breadth and depth) which are already available in real world. For example. A bagel can mean a lot depending on the power of DSL. A cooking DSL or scientific DSL or a super GDSL, which can handle all.

*. Accurately identifying the task when modeled by the user 3 times following on the lines of the said principle.

*. Learning a logic which can achieve what the user wants. Probably this part is not just the responsibility of DSL depending on the complexity of the features offered.


The inspiration for the above idea is from a MS windows program which was not successful but shipped as part of it during windows 95 (?) time. I had read about it long back but could not get the details now when searched. May be some of you, who have identified can let me know the exact idea and/or the link. I have not played second world but seen screenshots. It also might have influenced.


2 cents. thanks.

Edit: Appending this line:
Correction in above post. It is “second life” game, not “second world”. Sorry.

Edit2: clarification:
“limited to virtually infinite”. I meant “limited mappings” to “virtually infinite mappings”.

Provided that my answer is biased by my twenty years of experience in developing visual languages and the graphical frameworks needed to support them, I am a strong advocate of visual programming.

As you recognize, visual programming brings some opportunities for learners and also for domain experts but, I think, senior developers and designers are the ones who could get the greatest benefits in order to keep up with the complexity of software projects.

I agree that once you have a good metamodel and semantics the visual notation can be replaced by a textual one and still take advantage of most of the benefits of the language.
But you have to arrive to that point and, as you know, the process of designing a language is very iterative.

What I claim, based on my experience, is that visual programming makes it easier for you to climb the path of raising the level of abstraction.
Of course you must be able to evolve both the visual notations and the metamodels in order to be inspired; a fixed visual notation like Scratch, for instance, is useless and distracting to this end.

2 Likes

As you pointed out, and I haven’t highlighted in my first reply, the concrete notation (whatever textual or visual) should be a secondary concern.

If we share the goal of increasing the level of abstraction, as we do, the first step is to move away from concrete notation concerns so we can focus on the metamodel and semantics.

As for language development tools, this means for me that the grammar should be regarded as a secondary way of defining a language and that the definition of a metamodel should be enough for your tool to give you a working language that can be used with a generic notation and persistence (better if you get multiple of them for free).

And guess what, even in generic notations my preference goes to visual ones :slightly_smiling_face:

3 Likes

Some arguments in the article cited are not intrinsically related to languages being visual or not: plenty of textual languages exist which are already poorly designed on the abstract level. As @solmi.riccardo and @rafael already pointed out: most of the language is in its abstract syntax and its semantics, not in any concrete syntax.

Basically, the only half-way valuable argument from the article is that you can inspect and edit textual programs (or models) in any editor. For visual programs (which I assume are persisted in some kind of serialised AST format) this is not the case, and this is more often problematic than we care to admit, I think. It would be beneficial to be able to ship some kind of highly-portable viewer with a visual language so that you can ship visual programs without needing to ship the entire DSL implementation.

I’ve worked on something like that at Mendix: taking a visual program essentially representing a flowchart, and rendering that in a standalone form (SVG). By the way: Mendix is a good counter-example to visual languages not having a proven worth. In my current job I’m using MPS, and it’s a problem there as well, if only since it forces our users to share screenshots, and enough meta data (Git repo, branch, location, Node URL, etc.) when reporting problems. Some kind of standalone (localised) export would help a lot there.

1 Like

I would like it too!

Developing this tool requires a lot of effort and it would be nice to see if in this community we can find someone interested in collaborating on such a project.

Let me point out a few requirements and considerations:

  • Pluggable persistences to be able to define the formats of the models we want to read

  • Generic notations to be able to visualize models out of the box

  • Pluggable notations to be able to define specific notations for the models we are interested in

  • A persistence format suitable for distributing a model with its own specific notation

  • A discovery service to be able to distribute a model and get its own notation elsewhere

  • Unfortunately “highly-portable” cannot be a constraint for the Tool itself because a large part of its implementation is technology dependent.

  • Instead, “highly-portable” can be a constraint for the DSLs used to define persistences and notations

There are multiple graphical language workbenches that already have the basic requirements I have listed above and furthermore some authors/contributors are also in this community.
Unfortunately, a LWB has too many features and ends up looking more like an IDE than a model viewer even if you customize it.
In addition, each LWB has its own technological legacies which may make it unsuitable for this purpose.
For instance mine, the Whole Platform, is still based on Java/Eclipse.

I have been working on a Swift/SwiftUI port for years now. If someone is interested, you are welcome otherwise we can share something at the domain level.

1 Like

I think a portable, standalone, uneditable representation of models is something that could be valuable for Markus’ proposal as well: an Active Repository containing “any” model, irrespective of its client tooling, might be a bit hard to inspect outside of that client tooling, since its highly-structured data.

I’m guessing this essentially boils down to choosing a suitable rendering format: one that’s widely portable (e.g., HTML/SVG, or PDF), is resolution-independent, doesn’t take up much bandwidth, and can contain navigable anchors. To me, an MVP might be aiming as low as generating some HTML-based rendering (possibly including some SVG) of a section of a model. Choosing a rendering format might feel cheap, but defining meta languages to define model visualisations with achieves the same (potentially at a slightly higher level of abstraction), and you have to implement the visualising mapping in any case.

I used them as generator targets but I feel that they are a good choice only if you also choose Javascript/Typescript as your primary target language.

I know that the majority of you including Markus see this as a natural choice but I disagree.
I think that current web technologies will not give us not even the 15 years of write once and run everywhere that gave us Java/Eclipse.

I have made the choice of Swift / Swift UI as generator targets, I am still prototyping and engineering my Tool for them but I am very confident to be able to get almost for free also a web version.
May be that even Apple will announce a web porting this june.

I really meant pure HTML+CSS+SVG, just to transport a fully pre-rendered visualisation of model content. I agree you shouldn’t rely on {Java|Type}Script for things like that. I suspect a Web version of Swift(UI) could use the same approach. After all, HTML+CSS+SVG have been doing write once and run everywhere since before 1995, which is 25+ years :wink:

I also agree that current Web technologies won’t last for 15 years, but I’m not really concerned by that: as long as persisted models are still readable by then, I’m OK to use the technology of the mid-2030’s to make tools for those models.

Are you aware of graphviz and .dot grammar?
https://www.graphviz.org/

I am, but not to the extent that I know it well enough to see that/how it could completely replace HTML+CSS+SVG. I’m also not sure whether it’s really as portable across multiple OSs.

I guess PDF would currently be the most portable, but the problem with PDF is that it’s not as embeddable as HTML, plus you need plenty of external stuff (libraries) to produce PDF.

well any .dot file can be exported to pdf or png or svg, and then place the .svg inside a small html template.

I did use windows version to perform the “conversion” but .dot files can be created on any OS, are text based and grammar is very simple.

1 Like

To be honest, I don’t know any technology stack that has worked for me for more than one year without wasting a lot of time to just keep my project running on the updated version of the stack.

Eclipse breaks something on the Java version of my project every year. Now every three months.

The prototype of the Whole Platform for Web I managed to build 5 years ago with TypeScript/React/WebSockets and a few other components/styling/CI dependencies has accumulated so many problems in 5 years of lack of maintenance on my part that starting over would probably cost less effort.

Even the one-year old Swift/SwiftUI technology has already proved to be a great challenge. :slight_smile:

I really don’t understand how people manage to keep big projects alive without a language workbench that helps mitigate all efforts.
Is having a large amount of active contributors a viable alternative? (I never tried this one).

2 Likes

This is the reason corporations hire armies of coders to maintain their old stacks.

2 Likes