Visual Programming Should Start in the Debugger
Just like CAD didn't make everyone an industrial designer, visual programming isn't going to make everyone a programmer. Whether textual or visual, there'd still be a lot of underlying concepts (and how they are composed) to learn. But still, I think it's underexplored.
Most visual programming paradigms focus on coding--the instruction of the computer. This can be helpful for beginners, but it's of limited value to working programmers. Once syntax and API are learned, current visual tools don't help solve problems that working programmers have.
As "bicycles for the mind", computers should leverage our visual-spatial reasoning. It's a powerful aspect of our minds underutilized in modern programming toolchains and stacks. We all know this on some level, hence the gravity of the visual programming tarpit.
This is what debuggers are suppose to help with, but they don't have a good affordance. You can overstep the bug and need to restart. You have to manually track variables for your bug. And you certainly don't get a visual rep of your data structure and its changes over time.
Visual-spatial reasoning has a great affordance for illustrating states over time. Visual elements, like data, are inert and declarative. They describe things, not behavior. Hence, I think visualization should start with state and data, rather than code and behavior.
That's why I think visualization efforts should start with replacing the debugger, not the text editor. If you insist on the text editor, at least start with visually augmenting the text editor.
We have lang servers for IDEs; why not production statistics servers? Highlight a section of code with previous security breaches. Show the number of bugs with this section of code. Highlight hot paths in production. Show sample production data that flows through this code.
But that's just a side path. I think debuggers are the beachhead for visual programming. It's there that we can learn to visually express state for common data structures. And then learn how to visually express this state over time to express change.
Like check out the Hovenia Editor that visualizes trees. It's a good start to exploring visualizations of state in a debugger.
That said, compiler and language writers currently don't have this goal in mind, so the API of a compiler doesn't make it easy. So if you're going to write a new compiler for a new language, I'd make it easy to access the program state for visualization. Traditionally, it's was also difficult to find legible visual representations of state that adjusts to the problem at hand. A dict doesn't always want to be a key/value store. Sometimes they're histograms. Sometimes, they're a set. Perhaps LLMs or Edward Tufte can help. Once we have a visual language for illustrating state over time, I think we can utilize that visual language to back into visual programming. But until then, we're shoehorning current paradigms that don't have the right affordances to leverage our visual-spatial reasoning.
Visual programming might just look completely different than textual programming. Textual programming takes its cues from step-by-step (imperative), transformations (functional), inferences (logical) reasoning, all analytical traditions. It's rare to see visual programming take its cues from logic programming even though they have the same affordances. Declare a beginning and end state, and have the computer infer everything that needs to happen in between. Perhaps we visually express two states, a before and after, visual programming infers the programming instructions needed to take it from one state to another. Like drawing keyframes, and offloading the in-betweens to the computer.
But I don't think this is the only path forward. Most visual programming thinks from the top-down--from the affordances to humans down to the computer. There's a different way. How about instead of linear memory, we start with spatial memory at the bottom, and work our way up?
