I don’t really care about vibe coders but as a dev with just under 2 decades in the field:
Your vibe coding shit will not go to prod until humans fully review it
You better review it yourself first before offloading that massive mental drain to someone else (which means you still need to have some semblance of programming skills). Don’t open a PR with 250 files in it and then tell someone else to validate it.
Use more context. Don’t give it vague ass prompts.
Don’t use auto-accept. That’s just lazy asshole shit.
I can’t stress this enough: if you give me a PR with tons of new files and expect me to review it when you didn’t even review it yourself, I will 100% reject it and make you do it. If it’s all dumped into a single commit, I will whip your computer into the nearest body of water and tell you to go fish it out.
I don’t care what AI tool wrote your code. You’re still responsible for it and I will blame you.
Lmao glad I could help! I hate those big commits. They’re so much harder to traverse and know what’s going on. Developer experience has been big on my mind lately. Working 5 days a week is already hard, but there are moments when we can make tiny bits easier for each other.
I have never used an AI to code and don’t care about being able to do it to the point that I have disabled the buttons that Microsoft crammed into VS Code.
That said, I do think a better use of AI might be to prepare PRs in logical and reasonable sizes for submission that have coherent contextualization and scope. That way when some dingbat vibe codes their way into a circle jerk that simultaneously crashes from dual memory access and doxxes the entire user base, finding issues is easier to spread out and easier to educate them on why vibe coding is boneheaded.
I developed for the VFX industry and I see the whole vibe coding thing as akin to storyboards or previs. Those are fast and (often) sloppy representations of the final production which can be used to quickly communicate a concept without massive investment. I see the similarities in this, a vibe code job is sloppy, sometimes incomprehensible, but the finished product could give someone who knew what the fuck they are doing a springboard to write it correctly. So do what the film industry does: keep your previs guys in the basement, feed them occasionally, and tell them to go home when the real work starts. (No shade to previs/SB artists, it is a real craft and vital for the film industry as a whole. I am being flippant about you for commedic effect. Love you guys.)
I think storyboards is a great example of how it could be used properly.
Storyboards are a great way for someone to communicate “this is how I want it to look” in a rough way. But, a storyboard will never show up in the final movie (except maybe fun clips during the credits or something). It’s something that helps you on your way, but along the way 100% of it is replaced.
Similarly, the way I think of generative AI is that it’s basically a really good props department.
In the past, if a props / graphics / FX department had to generate some text on a computer screen that looked like someone was Hacking the Planet they’d need to come up with something that looked completely realistic. But, it would either be something hand-crafted, or they’d just go grab some open-source file and spew it out on the screen. What generative AI does is that it digests vast amounts of data to be able to come up with something that looks realistic for the prompt it was given. For something like a hacking scene, an LLM can probably generate something that’s actually much better than what the humans would make given the time and effort required. A hacking scene that a computer security professional would think is realistic is normally way beyond the required scope. But, an LLM can probably do one that is actually plausible for a computer security professional because of what that LLM has been trained on. But, it’s still a prop. If there are any IP addresses or email addresses in the LLM-generated output they may or may not work. And, for a movie prop, it might actually be worse if they do work.
When you’re asking an AI something like “What does a selection sort algorithm look like in Rust?”, what you’re really doing is asking “What does a realistic answer to that question look like?” You’re basically asking for a prop.
Now, some props can be extremely realistic looking. Think of the cockpit of an airplane in a serious aviation drama. The props people will probably either build a very realistic cockpit, or maybe even buy one from a junkyard and fix it up. The prop will be realistic enough that even a pilot will look at it and say that it’s correctly laid out and accurate. Similarly, if you ask an LLM to produce code for you, sometimes it will give you something that is realistic enough that it actually works.
Having said that, fundamentally, there’s a difference between “What is the answer to this question?” and “What would a realistic answer to this question look like?” And that’s the fundamental flaw of LLMs. Answering a question requires understanding the question. Simulating an answer just requires pattern matching.
I think this is great. I like hearing about your experience in the VFX industry since it’s unfamiliar to me as a web dev. The storyboard comparison is spot on. I like that people can drum up a “what if” at such a fast pace, but vibe coders need to be aware that it’s not a final product. You can spin it up, gauge what works and what doesn’t, and now you have feasibility with low overhead. There’s real value to that.
Edit: forgot to touch on your PR comment.
At work, we have an optional GitHub workflow that lets you call Claude in a PR and it will do its own assessment based on the instructions file we wrote for it. We stress that it’s not a final say and will make mistakes, but it’s been good in a pinch. I think if it misses 5 things but uncovers 1 bug, that’s still a win. I’ve definitely had “a-ha” moments with it where my dumb brain failed to properly handle a condition or something. Our company is good about using it responsibly and supplying as much context as we possibly can.
I like your previs analogy, because that’s how I’ve been thinking of it in my head without really knowing how to communicate it. It’s not very good at making a finished project, but it can be useful to demonstrate a direction to go in.
And actually, the one time I’ve felt I was able to use AI successfully was literally using it for previs; I had a specific idea of design I wanted for a logo, but didn’t know how to communicate it. So I created about a hundred AI iterations that eventually got close to what I wanted, handed that to my wife who is an actual artist, told her that was roughly what I was thinking about, and then she took the direction it was going in and made it an actual proper finished design. It saved us probably 15-20 iterations of going back and forth, and kept her from getting progressively more annoyed with me for saying “well… can you make it like that, but more so?”
I don’t really care about vibe coders but as a dev with just under 2 decades in the field:
I can’t stress this enough: if you give me a PR with tons of new files and expect me to review it when you didn’t even review it yourself, I will 100% reject it and make you do it. If it’s all dumped into a single commit, I will whip your computer into the nearest body of water and tell you to go fish it out.
I don’t care what AI tool wrote your code. You’re still responsible for it and I will blame you.
When I see a sloppy PR I remind people “AI didn’t write that. You wrote it. Your name is on the git blame.”
Love it, I have a vibe coding colleague I will use this with.
I like this mentality. I might start telling people the same thing
I’m going to steal this for an update to an internal guidance document for my dev team. Thank you.
Lmao glad I could help! I hate those big commits. They’re so much harder to traverse and know what’s going on. Developer experience has been big on my mind lately. Working 5 days a week is already hard, but there are moments when we can make tiny bits easier for each other.
I have never used an AI to code and don’t care about being able to do it to the point that I have disabled the buttons that Microsoft crammed into VS Code.
That said, I do think a better use of AI might be to prepare PRs in logical and reasonable sizes for submission that have coherent contextualization and scope. That way when some dingbat vibe codes their way into a circle jerk that simultaneously crashes from dual memory access and doxxes the entire user base, finding issues is easier to spread out and easier to educate them on why vibe coding is boneheaded.
I developed for the VFX industry and I see the whole vibe coding thing as akin to storyboards or previs. Those are fast and (often) sloppy representations of the final production which can be used to quickly communicate a concept without massive investment. I see the similarities in this, a vibe code job is sloppy, sometimes incomprehensible, but the finished product could give someone who knew what the fuck they are doing a springboard to write it correctly. So do what the film industry does: keep your previs guys in the basement, feed them occasionally, and tell them to go home when the real work starts. (No shade to previs/SB artists, it is a real craft and vital for the film industry as a whole. I am being flippant about you for commedic effect. Love you guys.)
I think storyboards is a great example of how it could be used properly.
Storyboards are a great way for someone to communicate “this is how I want it to look” in a rough way. But, a storyboard will never show up in the final movie (except maybe fun clips during the credits or something). It’s something that helps you on your way, but along the way 100% of it is replaced.
Similarly, the way I think of generative AI is that it’s basically a really good props department.
In the past, if a props / graphics / FX department had to generate some text on a computer screen that looked like someone was Hacking the Planet they’d need to come up with something that looked completely realistic. But, it would either be something hand-crafted, or they’d just go grab some open-source file and spew it out on the screen. What generative AI does is that it digests vast amounts of data to be able to come up with something that looks realistic for the prompt it was given. For something like a hacking scene, an LLM can probably generate something that’s actually much better than what the humans would make given the time and effort required. A hacking scene that a computer security professional would think is realistic is normally way beyond the required scope. But, an LLM can probably do one that is actually plausible for a computer security professional because of what that LLM has been trained on. But, it’s still a prop. If there are any IP addresses or email addresses in the LLM-generated output they may or may not work. And, for a movie prop, it might actually be worse if they do work.
When you’re asking an AI something like “What does a selection sort algorithm look like in Rust?”, what you’re really doing is asking “What does a realistic answer to that question look like?” You’re basically asking for a prop.
Now, some props can be extremely realistic looking. Think of the cockpit of an airplane in a serious aviation drama. The props people will probably either build a very realistic cockpit, or maybe even buy one from a junkyard and fix it up. The prop will be realistic enough that even a pilot will look at it and say that it’s correctly laid out and accurate. Similarly, if you ask an LLM to produce code for you, sometimes it will give you something that is realistic enough that it actually works.
Having said that, fundamentally, there’s a difference between “What is the answer to this question?” and “What would a realistic answer to this question look like?” And that’s the fundamental flaw of LLMs. Answering a question requires understanding the question. Simulating an answer just requires pattern matching.
I think this is great. I like hearing about your experience in the VFX industry since it’s unfamiliar to me as a web dev. The storyboard comparison is spot on. I like that people can drum up a “what if” at such a fast pace, but vibe coders need to be aware that it’s not a final product. You can spin it up, gauge what works and what doesn’t, and now you have feasibility with low overhead. There’s real value to that.
Edit: forgot to touch on your PR comment.
At work, we have an optional GitHub workflow that lets you call Claude in a PR and it will do its own assessment based on the instructions file we wrote for it. We stress that it’s not a final say and will make mistakes, but it’s been good in a pinch. I think if it misses 5 things but uncovers 1 bug, that’s still a win. I’ve definitely had “a-ha” moments with it where my dumb brain failed to properly handle a condition or something. Our company is good about using it responsibly and supplying as much context as we possibly can.
I like your previs analogy, because that’s how I’ve been thinking of it in my head without really knowing how to communicate it. It’s not very good at making a finished project, but it can be useful to demonstrate a direction to go in.
And actually, the one time I’ve felt I was able to use AI successfully was literally using it for previs; I had a specific idea of design I wanted for a logo, but didn’t know how to communicate it. So I created about a hundred AI iterations that eventually got close to what I wanted, handed that to my wife who is an actual artist, told her that was roughly what I was thinking about, and then she took the direction it was going in and made it an actual proper finished design. It saved us probably 15-20 iterations of going back and forth, and kept her from getting progressively more annoyed with me for saying “well… can you make it like that, but more so?”