AI is slowly making its way into every aspect of our day-to-day lives. Hey Google, set a timer for 30 minutes. Siri, remind me to pay the bills on Monday. Alexa, schedule a meeting at 3PM tomorrow. It’s becoming more and more common to delegate these simple admin and memorization tasks to our digital assistants, so that our brains can solve other, more important and oftentimes resource-intensive problems at (relative) peace.

Software engineers are no different in this sense – if anything, they are usually even more likely to be the first in getting their hands on some tech novelties and trying them out for themselves – be it for fun, work, or both. And so, we decided to conduct a little investigation to see what the situation is like at Noveo: what do our developers think about different AI tools – more specifically, about AI coding assistants? Do they use any in their daily work? And have they noticed any change in their productivity levels since adopting some of these virtual helpers? Let’s discuss!

“Let me get this one thing clear: AI will not replace developers, at least not in the foreseeable future,” this is how most of our interviews begin – or end – no matter the solution we discuss. A bit of a spoiler, to be honest, but quite a good news nonetheless in case you were wondering whether digital creations were able to defeat their human creators ;)

Recalling our previous conversation with the Noveo design team (which ultimately came to the same conclusion), we ask, “But is there a chance of coexistence? Can all these tools be considered your helpers, or are they more of a nuisance despite being called assistants?” And this is where opinions start to differ… :) 

“Naturally, everyone’s answer will depend on the tools they use, but it’s also about the how, the why, and the what for,” says one of the interviewees. It’s hard to disagree with them – so, why not take a look at a couple of the most popular AIs among our engineers and check how useful they truly are when it comes to real-life projects and tasks?

GitHub Copilot

Trained on countless lines of code, GitHub Copilot is one of the most powerful generative AI tools to date. A simple Google search will provide you with millions of articles describing its innovative features and capabilities – but do our developers agree with all of those praises? Here’s what they had to say:

“When you first start using Copilot, you can’t help but go, ‘Wow, that’s so cool!’ It fully integrates into your IDE, reads and analyzes the code you’re writing in real time, and suggests possible solutions. It’s incredibly useful and really helps to boost your coding speed.”

“Copilot is great for working on routine tasks, especially when it comes to writing boilerplates – and yes, sometimes those are necessary, so having a tool that autocompletes some chunks of code for you is a huge help.”

“It’s good at predicting what you need to write next based on the context. It can also fully implement certain functions for you – you just need to write the prompt, or the function’s name, correctly. However, I’ve noticed that it only works for some rather plain and uncomplicated tasks – anything more complex is too difficult for Copilot to process.”

“One of the things I really liked about Copilot is how it handles same-type objects. In my case, it surprised me with translations: I had a file with multiple elements that consisted of a default message in French and an ID in English, with IDs being sort of a shortened version of a French phrase. Apparently, Copilot understood the idea and, when I began adding new elements, it quickly came up with complete sentences based on the English ID alone! No need to distract our French colleagues, and also, no need for the customer to write anything from scratch – all that was left for them was to validate the AI’s suggestions. Quite a neat feature, and there are lots of similar ones too – you just need to approach them with the right kind of mindset.”

“Since Copilot analyzes the whole project, it can help the team to stick to one single style, and write a code that is more uniform from the start. It also suggests different names for elements when you can’t decide on what to call them. All in all, it allows you to focus on more difficult things.”

Looks great so far! Any objections?

“From what I gather, Copilot is good at doing what it was created to do – autocompleting some common functions, and finding the most suitable solutions on developer forums like StackOverflow or GitHub Discussions. But when you work on actual projects, tasks that you can fully delegate to AI – and get a meaningful result in return – are very few and far between.”

“At one point I realized that I was spending more time on trying to understand Copilot’s suggestions – which never stopped coming, by the way – than on writing actual code.”

“More often than not, it’s better to just write something from scratch yourself – otherwise, you risk getting sucked into a black hole of endless refactoring. Even manually searching for something specific on StackOverflow and then adapting the solution to your needs would take considerably less time.”

“I’ve noticed that Copilot uses quite a lot of CPU resources, which also slows down other programs and processes. So now not only are you distracted by its suggestions but can’t work on anything else either.”

Got it. As is the case with anything else in life, everything is good in moderation :) Use Copilot for simple and straightforward tasks to gain some time and save your brainpower, and then apply these new-found resources to tackle challenging and non-trivial issues. Win-win!

Tabnine

Although Tabnine is a bit of a less popular tool, it is generally considered to be quite a nice alternative to Copilot. Is it actually so in practice?

“I had a chance to use Tabnine, Copilot, and CodeWhisperer in my work – whenever I joined new projects, the teams would usually have an established set of tools, so I used that as an opportunity to try out and learn new solutions. I’d say, all of them left quite a positive impression, but it’s true that you’d still have to look through any AI-generated code, no matter the source – for example, to check it for bugs or correct handling of edge cases.”

“I’d say that Tabnine has been a big contributor to my working speed. It guesses what you’re planning to type next – taking into account not only the programming language you’re using but also the context of your comments, etc. – so all you have to do is press Tab and keep coding. Even though its suggestions aren’t always right, it’s still a good tool to keep handy in general.”

“Admittedly, Tabnine isn’t as bright when it comes to serious, real-world tasks; on the other hand, it’s not what it’s there for in the first place. It spares you all the mundane and routine operations, fast forwarding you to the truly fun and exciting parts. And this is exactly what I want from a tool like this.”

“It’s good at providing specific, ‘bite-size’ tips, like when you need to finish a line or a long element name. For instance, one of my projects had a lot of nested components and quite an intricate system of classes overall. Interestingly, Tabnine was able to suggest the right ‘endings’ off the bat – and so, ca… would quickly turn into car.engine.valve.3, without skipping any middle parts. The example is rather simple but you get the idea – it allows you to gain a bit of time and gets rid of potential mistakes.”

“We’re used to talking about all these cases where developers need to fix different mistakes in AI’s code, but I think it’s more fun to discuss the times when things went in a somewhat opposite direction. For example, I remember a couple of suggestions from Tabnine that actually made me realize why something I had been writing wasn’t necessarily the best solution – or rather, not the most concise one. Sometimes you forget that keeping things simple is also an option.”

“To be fair, I haven’t used Tabnine on any Python-based projects yet, but I think that it’s going to be an even better helper there. Considering the fact that Python is its ‘mother tongue’, as well as the strict and precise nature of the language itself, it’s bound to show great results. Then again, if it’s not the case, I’m sure that it would still prove to be a really useful tool – because, unlike humans, it can actually ‘remember’ all of the countless libraries and ML-specific functions.”

Okay, the advantages are clear. What about the downsides?

“Unfortunately, in my experience, Tabnine was helpful only when the tasks were super common, simple and linear – so much so that, whenever it made a correct suggestion, I would catch myself thinking, ‘I must be writing such cliché things right now and I haven’t even realized it.’”

“It’s nothing new, but is still worth mentioning: each AI needs to be trained, again and again, in order to adapt to the user and whatever they’re working on. Sometimes it may be quick and easy, and at other times – not so much. It’s just like project onboardings we’re all familiar with.”

“I guess it depends, but personally, I’ve noticed how easy it is for me to get distracted when using assistant tools. Instead of a metaphorical blank page, you constantly see some ‘future’ code chunks and unwillingly shift your attention to check if any of them are right. Like so, you spend some time on reading through all these ongoing suggestions, and then you spend some more trying to remember what you actually wanted to write… Rinse, repeat.”

So, not that much different from Copilot reviews – which means that both tools can be used interchangeably, but only for routine, almost-autopilot operations. Good to know! :)

Food for thought

Actually, the problem of focus switching raised by some of our developers is quite an interesting one – while one may write it off as simple inattentiveness or a mild case of ADHD (who knows?), it is, in fact, a very natural and common thing our brain does.

As shown by research, human brain is designed in a way that keeps it on the constant lookout: we only truly concentrate for short bursts of time, and shift our focus in-between in order to scan the environment, understand what’s going on around us, and continue on with the task at hand. In prehistoric times, this ‘distractibility’ allowed our ancestors to spot approaching predators and avoid danger, whereas nowadays it acts more as a screen which filters out any low-priority events so that we can then redirect our attention to whatever is more important at the moment.

Most of the time we don’t even realize that it’s happening, however, there may be cases where going back to work takes us a (noticeable) while. This phenomenon is referred to as switch cost and is also a very normal thing – essentially, it means that our brain needs some time to recollect itself after an interruption and ‘change lanes’ of thought. Makes sense, right?

So, what does it mean in the context of programming and using various real-time ‘helpers’? Some may argue that having any sort of hints on their screen is too distracting and borderline intrusive, while others may get fully accustomed to this ‘addition’ and only pay attention to AI’s recommendations when needed. In all fairness, either side is right in their own way – after all, it all boils down to one’s preferences and work habits, as well as individual particularities in terms of information processing.

Conclusion

Having looked at AI tools and weighed all of their pros and cons together with our development team, one thing we can say for certain is this: although modern technologies are far from being as sharp-witted as humans, it is well worth the try to take them on board and make them work with and for you :)

First, remember that the purpose of coding assistants is to not write everything for you but automate some parts of the most common, standard actions you do. Then, pick and mix a couple of solutions to find a combination that works best for you, and have fun! For example, we’ve come to the conclusion that Tabnine shows better results when used for autocompleting elements within a line or two, while Copilot seems to be more ‘global context’-aware and thus may come in handy when working on same-type objects or fixing up some styling, amongst other tasks.

Even though this kind of help may not seem as much at first, in reality, it accumulates over time and can result in a significant productivity boost – our stats show an increase of development speed up to 15-20%, with some engineers’ reporting an over 50% difference in numbers!

Of course, getting distracted to the point of actually slowing down and forgetting important details is a valid concern and also a possibility, but we firmly believe that it never hurts to experiment and discover new things for yourself. Some solutions may work out and others may not – but guess what? You’ll never know until you give it a try ;)

Once again, a big thank you to all of our interviewees – it was great to hear your opinions on the matter! Let’s keep the conversation going, and see where we land it a couple of years?