It's not OCR. Those colored dots along the periphery are fiducials. They're how Realtalk recognizes a program.
Every object that you can interact with in Realtalk has a unique set of fiducials. When the camera sees one, it looks up its behavior and projects it accordingly.
Correct, that doesn't work in the system as implemented. But my understanding is that there has been a goal from the start to _eventually_ get physical code editing to work. I recall hearing the researchers talking about wanting to eventually get handwriting recognition working, so you could edit code by hand.
Cameras good enough to do that kind of work aren't really available right now. It does still make the code more accessible by making it immediately visible on the object so if you want to know how a thing works you can just pick it up.
... so in the intro video at like 2:37, it says that a program can be in language, or an arrangement of components, or a hand-drawn diagram, because anything you can write an interpreter for is a program. The "arrangement of components" and the "hand-drawn diagram" shots show stuff drawn on a whiteboard, which seems like it can be edited by redrawing. Are you saying that a hand-drawn diagram program can be freely edited by changing the diagram, but a language-based program cannot be edited by changing the representation of its code? That seems ... worse.
I would think that you'd want a fiducial marker on code-printouts to identify that a piece of code has a language and a namespace perhaps (i.e. "this page contains a kotlin definition that lives in com.acme.foo"), but that the contents should be modifiable. Otherwise, what's the point? If to edit the code you're gonna pull up a keyboard and screen (or project an editor window) ... then this seems like what we already have, plus you have to print stuff out.
It's been 6 years since I've been there. I'm sure they've made developments since then.
I also think these comments along the lines of "what about X" are unhelpfully reductivist/dismissive. The whole point is that it's a research project. If you can think of a better way to make ephemeral room-sized computing work - cool, let's try that! Just because it worked some way when I was there in 2018 or some way in the video doesn't mean that's the end vision for how it will always be.
This isn't a product. It's a vision for the future.
> The whole point is that it's a research project.
> It's a vision for the future.
Neither of these mean we can't or shouldn't be able to have discuss whether parts of it are good or bad, make more or less sense. Is it a vision of a future you'd want to work with/in?
> I also think these comments along the lines of "what about X" are unhelpfully reductivist/dismissive.
My first statement was "I think the overall idea here is really cool". The intent is not to be dismissive. But if you think the only acceptable reaction is unalloyed praise ... then why even have it on a discussion-oriented site?
I think the way of working being demonstrated seems like a great fit for some kinds of work and that trying to awkwardly shoehorn software-development to happen in their system detracts rather than adds to it.
> If you can think of a better way to make ephemeral room-sized computing work
... I think an IDE, a keyboard, and a projector are better than printing code blocks at a specific revision which is identified by a computer-readable id, and which must be given a new ID and a new printed page every time you want to try executing a new version.
Yours wasn't the only comment along those lines, and I was replying to the class of them rather than yours specifically.
I don't mean to curtail discussion or say that only praise is allowed. I just want to steer away from "gotcha" energy on a research project.
Ideas/discussion/critique are welcome! "This project is dumb because it does things differently than I'm used to" or "because it currently only supports digital changes to the physical paper" are less helpful. Part of the fun of a research project is trying weird stuff to see what feels better and what doesn't.
Again, none of this is directed at you personally or about your specific comment. I just noticed a trend of comments about the code editing experience that felt more like trying to dunk on the concept than promoting curious discussion.
> I think an IDE, a keyboard, and a projector are better than printing code blocks at a specific revision which is identified by a computer-readable id, and which must be given a new ID and a new printed page every time you want to try executing a new version.
You're making several incorrect assumptions here:
1. That you can't interactively try out the code as you're editing it.
2. That the system as implemented is the final vision of how the system ought to work forever.
From https://youtu.be/5Q9r-AEzRMA?t=150 "Anyone can change any program at any time and see the changes immediately", which demonstrates live editing the code and seeing the projected flow lines change color. So you can keep editing and iterating on the program, trying out your changes without ever having to print anything. Once you are satisfied with your improvements, you then "commit" the code, which results in the system printing out a new page with its new identifier.
And if any part of your expectations isn't how things work, it's likely because this is a research project and nobody has written the code to make it behave the way you'd like. Since Realtalk is built on itself, one would only need to make the appropriate changes to improve the system.
Every object that you can interact with in Realtalk has a unique set of fiducials. When the camera sees one, it looks up its behavior and projects it accordingly.