At $WORK, we have a bot that integrates with Slack that sets up minor PRs. Adjusting tf, updating endpoints, adding simple handlers. It does pretty well.
Also in a case of just prose to code, Claude wrote up a concurrent data migration utility in Go. When I reviewed it, it wasn't managing goroutines or waitgroups well, and the whole thing was a buggy mess and could not be gracefully killed. I would have written it faster by hand, no doubt. I think I know more now and the calculus may be shifting on my AI usage. However, the following day, my colleague needed a nearly identical temporary tool. A 45 minute session with Claude of "copy this thing but do this other stuff" easily saved them 6-8 hours of work. And again, that was just talking with Claude.
I am doing a hybrid approach really. I write much of my scaffolding, I write example code, I modify quick things the ai made to be more like I want, I set up guard rails and some tests then have the ai go to town. Results are mixed but trending up still.
FWIW, our CEO has declared us to be AI-first, so we are to leverage AI in everything we do which I think is misguided. But you can bet they will be reviewing AI usage metrics and lower wont be better at $WORK.
Great time to research if those choices are still valid or if there's a better way. In any regard, its just an overview, not a total rewrite from the AI's perspective.
> it wasn't managing goroutines or waitgroups well, and the whole thing was a buggy mess and could not be gracefully killed
First pass on a greenfield project is often like that, for humans too I suppose. Once the MVP is up, refactor with Opus ultrathink to look for areas of weakness and improvement usually tightens things up.
Then as you pointed out, once you have solid scaffolding, examples, etc, things keep improving. I feel like Claude has a pretty strong bias for following existing patterns in the project.
> FWIW, our CEO has declared us to be AI-first, so we are to leverage AI in everything we do which I think is misguided. But you can bet they will be reviewing AI usage metrics and lower wont be better at $WORK.
I've taken some pleasure in having GitHub copilot review whitespace normalization PRs. It says it can't do it, but I hope I get my points anyway.
This is a great response, even for a blue collar worker understanding none of its complexities (I have no code creation abilities, whatsoever — I can adjust parameters, and that's about it... I am a hardware guy).
My layperson anecdote about LLM coding is that using Perplexity is the first time I've ever had the confidence (artificial, or not) to actually try to accomplish something novel with software/coding. Without judgments, the LLM patiently attempts to turn my meat-speak into code. It helps explain [very simple stuff I can assure you!] what its language requires for a hardware result to occur, without chastising you. [Raspberry Pi / Arduino e.g.]
LLMs have encouraged me to explore the inner workings of more technologies, software and not. I finally have the knowledgeable apprentice to help me with microcontroller implementations, albeit slowly and perhaps somewhat dangerously [1].
----
Having spent the majority of my professional life troubleshooting hardware problems, I often benefit from rubber ducky troubleshooting [0], going back to the basics when something complicated isn't working. LLMs have been very helpful in this roleplay (e.g. garage door openers, thermostat advanced configurations, pin-outs, washing machine not working, etc.).
what really comes through in this description is a fear of judgement from other people, which I think is extremely relatable for anyone who's ever posted a question on stack overflow. I don't think it's a coincidence that the popularity of these tools is coinciding with a general atmosphere of low trust and social cohesion in the US and other societies this last decade
On her deathbed, years ago, my beloved mother lamented that she often felt mentally bullied by her three brilliant sons [0], even decades into our adulthoods; embarassed, she would censor her own knowledge-seeking from the people she trusted most [2].
She didn't live long enough to use ChatGPT [1] (she would have been flabbergasted at its ability to understand people/situations), but even with her "normal" intelligence she would have been a master to its perceptions/trainings.
[0] "Beyond just teasing."
[1] We did briefly wordplay with GPT-2 right before she died via thisworddoesnotexist.com exchanges, but nothing conversive.
[2] Relavent example, to the best of my understanding of hers: I would never ask my brilliant engineer programmer hardwarebro for coding help on any personal project, never. Just as I don't ask lawyerbro for personal legal advice.
----
About a year later (~2023), my dentist friend experienced a sudden life change (wife sick @35); in his grieving/soul-seeking, I recommended that he share some of his mental chaos with an LLM, even just if to role-play as his sick family member. Dr. Friend later thanked me for recommending the resource — particularly "the entire lack of any judgments" — and shared his own brilliant discoveries using creative prompt structuring.
----
Particularly as a big dude, it's nice to not always have to be the tough guy, to even admit weakness. Unfortunately I think the overall societal benefits of generative AI are going to increase anti-social behaviour, but it's nice to have a friendly apprentice that knows something about almost everything... any time... any reason.
As a software guy going way back, this post may be the death knell of software development as I've known it. I have never seen a good hardware guy who could code his way out of a paper bag. If hardware guys succeed in developing software with LLM coding, then it's time to abandon ship (reaches for life preserver pension).
The risk is that lay people read comments like this and conclude "ergo, we need fewer programmers."
Nothing that the LLM is outputting is useful in the hands of somebody who couldn't have done it themselves (at least, given a reasonable amount of time).
The most apt analogy is that of pilot and autopilot. Autopilot makes the job of the pilot more pleasant, but it doesn't even slightly obviate the need for the pilot, nor does it lower the bar for the people that you can train as pilots.
The benefits of LLM programming are mostly going to be subsumed by the operator, to make their lives easier. Very little is gonna go to their employer (despite all the pressure), and this is not due to some principal-agent breakdown; it's just intrinsic to the nature of this work.
Where I am, headcount is based on "can we finish and sustain these planned and present required projects". If these automations allow a developer to burn less time, it reduces the need for headcount. As a direct result of this approach to hiring based on need, the concept of a "layoff" doesn't exist where I am.
>If these automations allow a developer to burn less time, it reduces the need for headcount.
This is exactly the fallacy, and it's very hard to see why it's a fallacy if you've never professionally written code (and even then).
Software development work fills to occupy the time allotted to it. That's because there is always a tradeoff between time and quality. If you have time available, you will fundamentally alter your approach to writing that piece of software. A rough analogy: air travel doesn't mean we take fewer vacations -- it just means we take vacations to farther away places.
Because of this effect, a dev can really finish a project in as little time as you want (up to a reasonable minimum). It just comes down to how much quality loss and risk can be tolerated. I can make a restaurant website in 1 hour (on Wix/Squarespace) or in 3 months (something hand-crafted and sophisticated). The latter is not "wasted time", it just depends on where you move the lever.
However, sometimes this is a false tradeoff. It isn't always necessary that the place you flew 3 hours will give you a better vacation than some place you could've driven to in 3 hours. You only hope it's better.
>As a direct result of this approach to hiring based on need, the concept of a "layoff" doesn't exist where I am.
LLMs or not, you could've just hired fewer people and made it work anyway. It's not like if you hired 3 people instead of 6 before the LLM era, it was impossible to do.
The gist of it is that LLMs are mostly just devs having fun and tinkering about, or making their quality of life better, or implementing some script, tooling, or different approach that they might've avoided before LLMs. There's no powertrain from that stuff to business efficiency.
Sorry, I didn't mean the "you" to be personal. It's a general "you".
But if you meant it's inappropriate even as a general statement then I disagree. Some concepts are just difficult to convey or unintuitive if one hasn't actually done the thing. It's more of a disclaimer that what's to follow is unintuitive.
> The benefits of LLM programming are mostly going to be subsumed by the operator, to make their lives easier. Very little is gonna go to their employer
your boss is going to let you go home if you get all your work done early?
I think your experience matches well with mine. There are certain workloads and use cases where these tools really do well and legitimately save time; these tend to be more concise tasks and well defined with good context from which to draw from. The wrong tasking and the results can be pretty bad and a time sink.
I think the difficulty is exercising the judgement to know where that productive boundary sits. That's more difficult than it sounds because we're not use to adjudicating machine reasoning which can appear human-like
... So we tend to treat it like a human which is, of course, an error.
I find ChatGPT excellent for writing scripts in obscure scripting languages - AppleScript, Adobe Cloud products, IntelliJ plugin development, LibreOffice, and others.
All of these have a non-trivial learning curve and/or poor and patchy docs.
I could master all of these the hard way, but it would be a huge and not very productive time sink. It's much easier to tell a machine what I want and iterate with error reports if it doesn't solve my problem immediately.
So is this AGI? It's not self-training. But it is smart enough to search docs and examples and pull them together into code that solves a problem. It clearly "knows" far more than I do in this particular domain, and works much faster.
So I am very clearly getting real value from it. And there's a multiplier effect, because it's now possible to imagine automating processes that weren't possible before, and glue together custom franken-workflows that link supposedly incompatible systems and save huge amounts of time.
Also in a case of just prose to code, Claude wrote up a concurrent data migration utility in Go. When I reviewed it, it wasn't managing goroutines or waitgroups well, and the whole thing was a buggy mess and could not be gracefully killed. I would have written it faster by hand, no doubt. I think I know more now and the calculus may be shifting on my AI usage. However, the following day, my colleague needed a nearly identical temporary tool. A 45 minute session with Claude of "copy this thing but do this other stuff" easily saved them 6-8 hours of work. And again, that was just talking with Claude.
I am doing a hybrid approach really. I write much of my scaffolding, I write example code, I modify quick things the ai made to be more like I want, I set up guard rails and some tests then have the ai go to town. Results are mixed but trending up still.
FWIW, our CEO has declared us to be AI-first, so we are to leverage AI in everything we do which I think is misguided. But you can bet they will be reviewing AI usage metrics and lower wont be better at $WORK.