I wouldn't trust those master prompt engineers until they start to release impressive projects. If this technology actually is such a huge productivity boost it shouldn't be too hard, so until that happens I take it as evidence that the productivity boost isn't that extreme.

It all sounds like a very well orchestrated marketing campaign, HN would be the first place I would seed with stories if I was promoting it as a dev tool.

I spent few hours asking to do few things with a well documented public but uncommon api. It sent me on a wild goose chase of functionality that API never had..(contacted developer to make sure I am not crazy). It wrote some stuff that looked VERY good, except for the fact that it interacted with things that were never there. Would have been nice if they were..

It seem to exercise wishful thinking. You can ask it to implement functionality and it will just imagine that API/system has stuff needed.

Major problem is that I for one have no clue how it is "thinking" maybe over time we will develop better understanding of it.

When it imagines stuff that isn't there, I ask it to implement the missing piece. Doesn't always work, but often does

Because different programmers face different problems. Surprise, I know.

For reference, I used ChatGPT to build a simple ChatGPT-clone web interface. The JavaScript it generated is literally 100% correct (not 100% good. By correct I mean it does what I describe in prompt, without syntax errors or calling non-existing functions. Code can be both correct and bad, obviously.)

I also tried to use ChatGPT to generate some Houdini vex code. It sometimes gave me a useful scaffold, but mostly just pure garbage.

Probably both, depending on which developers you compare against and how you measure productivity. Keep in mind there's an entire world of development where productivity is measured by number of JIRA tickets closed for button text updates.

It's great for boilerplate, like a snippets+. If I ask it to solve something that's basically straight out of the documentation / walk through, it spits that back at me. Bad at doing the hard work imo. Useless at solving novel problems, or those that don't have a solution already within its data set.

Being limited the way it is - not being able to use a project's greater context, which is millions of tokens generally - is a severely limiting factor that makes it unsuitable for serious work as well. Working around this (figuring out what dependencies to inject into the prompt) is as much work as actually coding.

I can see that it's passed some sort of threshold with the greater population. It is great generating filler text. Just what it can do right now should have many applications.

If you have tolerance for mistakes, it can generate first drafts extremely quickly, freeing up your time to debug them. If you give up after seeing an error, it won't work for you.

"Automating the tedious job of writing bugs so you can get on with the important business of investigating them," was a way I jokingly described the value proposition of AI coding assistants to colleagues in the early days of Copilot. It's amusing to see it stated unironically here.

Whether it saves time or not really depends on the task.

The other day, I had a subtitles file with slightly mismatched timestamps. GPT wrote me a Python script to fix them that got 90% of the way there (and, in particular, the code had all the API calls that I needed to get it to 100%, even though this is the first time I've heard of the library it used). The whole thing took less time than finding and installing the app that would do it for me.

The catch is that you need to have a pretty good "gut feel" understanding of its limitations to figure out whether it's going to be a time saver or not before you sink too much time into making it do something right. But it is a skill that can be learned from experience (for a particular model, anyway), and I suspect that the ability to decide what to delegate and what to do yourself will be one of the key differences between junior and senior devs going forward.

I feel similarly to the parent. It gives me a good rough first draft of the code I need, and since I'm using statically typed languages, the errors are generally pretty minor (not giving the right type annotations for example) that it still saves me a lot of time overall from writing everything from scratch.

As a mere mortal, I often write my own bugs, so I'm happy for a machine to write them for me so I can move on to the next step.

Agree. I’ve used it to help set up Bonjoir working across my home VPN. It wrote my the commands for EdgeOS but it kept insisting on adding commands that didn’t exist for my hardware version. I kept telling it “no that doesn’t work on my router” and it apologized and wrote the exact same thing again.

The good news is 80% of what it spat out was usable. I gave up getting it to try to give me the last 20% and figured it out myself.

One thing I’ve found helpful in those cases it to tell it that it doesn’t know what it’s talking about and ask it for good keywords to search on Google.

Someday I’d love if it just gave me the relevant links itself…

> "more productive than 10 developers put together"

As long as it can't read my entire codebase, understand it and help me with it - that's absolute horseshit. I don't spend much time writing a bunch of new code, I'm spending most of it trying to understand the heaps of legacy code in my company and then make some small tweaks/additions.

The day it can understand big code repos will truly be the end for human made code I think.

Both, it depends on the domain. For Rust work, it's pretty good, since Rust is stable enough to fit in within the 2021 cutoff. For Flutter, it's not that good due to the aforementioned cutoff and the changes in Flutter since then.

I used to get more non-existent APIs but not as much with GPT 4 recently.

I think it does help in some scenarios like small scripts or if you are learning something new. But often it adds an overhead. You have to check constantly wether or not to accept the suggestions.

I dont use copilot anymore (at least for now). Just chatgpt as an alternative to google/SO

Maybe not 10x, but it can certainly speed up certain projects for a senior developer, cutting through the boilerplate when starting. It's also excellent for optimisation.

You have to chain prompts to get good results, also the way you say things can be important. Think voice assistants requiring some nuance to their instructions

Here's an example we can all try (and help me out). Validating that the input into a date field is a valid date using formik and yup. The problem is that stuff like "11" gets turned into a valid date:

Abbreviated prompt:

  dateOfBirth: date().required().typeError('Invalid Date')

})

The dateOfBirth validation is incorrect because it accepts values like "11". What is the correct validation to ensure that a user has inputted a valid date?

GPTs first go:

**** dateOfBirth: date() .required() .typeError('Invalid Date') .min(new Date(1900, 0, 1), 'Date must be after 1900') .max(new Date(), 'Date cannot be in the future'),

In this example, the min method sets a minimum date of January 1, 1900, and the max method sets a maximum date of the current date. You can adjust the minimum and maximum dates as needed to fit your specific use case. ****

Me:

It still treats "11" as valid input

GPT second attempt:

***** dateOfBirth: date() .required() .typeError('Invalid Date') .test('valid-date', 'Invalid date', (value) => { if (!value) { return false; } const date = new Date(value); return !isNaN(date.getTime()); }), *****

Me:

The parameter 'value' passed into test is a date and not a string

GPT third attempt:

******

  dateOfBirth: date()
    .required()
    .typeError('Invalid Date')
    .test('valid-date', 'Invalid date', (value) => {
      return value instanceof Date && value.isValid();
    }),

******

Me:

There is no isValid function on Date objects

GPT fourth attempt:

********

  dateOfBirth: date()
    .required()
    .typeError('Invalid Date')
    .test('valid-date', 'Invalid date', (value) => {
      return value instanceof Date && !isNaN(value.getTime());
    }),

********

Me:

That still accepts 11 as a valid date.

GPT fifth attempt:

*********

  dateOfBirth: string()
    .required()
    .test('valid-date', 'Invalid date', (value) => {
      const dateRegex = /^(?:(?:31(\/|-|\.)(?:0?[13578]|1[02]))\1|(?:(?:29|30)(\/|-|\.)(?:0?[1,3-9]|1[0-2])\2))(?:(?:1[6-9]|[2-9]\d)?\d{2})$|^(?:29(\/|-|\.)0?2\3(?:(?:1[6-9]|[2-9]\d)?(?:0[48]|[2468][048]|[13579][26])|(?:(?:16|[2468][048]|[3579][26])00)))$|^(?:0?[1-9]|1\d|2[0-8])(\/|-|\.)(?:(?:0?[1-9])|(?:1[0-2]))\4(?:(?:1[6-9]|[2-9]\d)?\d{2})$/;
      if (!dateRegex.test(value)) {
        return false;
      }
      const date = new Date(value);
      return date instanceof Date && !isNaN(date.getTime());
    }),

********

Which may or may not be correct. That regex is bonkers and given the errors in the previous responses I don't trust it.

In my experience it works well with components that are not architectural and with vanilla languages rather than frameworks.