We’ve been running an experiment for several weeks sending suggestions by ChatGPT to users asking questions on the webMethods forum. Last week I asked @marielavd to disable that feature as most of the suggestions weren’t helpful.
Some issues we found and lessons we learned:
- The model wasn’t trained on our data (Software AG docs, Tech Community solved answers, KB, trainings, etc).
- We used ChatGPT 3.5. GPT-4 should be better, but still has many flaws.
- The OpenAI API still cannot understand images, so information provided in a screenshot wouldn’t be interpreted by the bot.
- We noticed the AI making up stuff like functionalities, details or even links.
- Often the bot would repeat the question in other words or ask for details that are already provided.
- Since often users don’t specify the version of their product the bot could provide answers for another version (that’s applicable to real people also, but usually our senior members would first ask for product details).
- It’s possible the AI to give a solution that works, but is insecure or causes performance issues.
We’ll continue playing around with AI and ways to leverage its power, but this time feeding it better data and perhaps having a man-in-the-middle.
I’m curious do you find any of these new tools helpful for technical tasks and do you use some regularly? I have deliberately skipped most of the coding tools like CodeWhisperer, since they are pretty advanced and have been around for some time (now they just got a bit more AI).
Which new tool do you find most helpful in technical tasks (programming, creating services, administration, finding information, debugging)?
- OpenAI/ChatGPT/GPT-4
- Claude/Anthropic
- Google Bard
- JasperChat
- AskCodi
- GitHub CoPilot
- None
- Other (please specify in the comments below).
0 voters