Maybe call it backseat programming for now?
GitHub on Tuesday unveiled a code-completion tool called Copilot that shows promise though still has some way to go to meet its AI pair programming goal. If you’re wondering how well it performs, in an FAQ about the service, which is available as a limited « technical preview, » GitHub admitted: And: Developer Nick Shearer spotted that one of the examples of Copilot’s output on its own homepage was the auto-completion of an algorithm that processes expenses. It suggests using a floating-point variable for a monetary value. Generally speaking, floating-point variables are not suited for currency, unless you want values like $9.99000001 going through your system. Copilot may find other ways to ruin your day. We’ll let GitHub explain: The most enthusiastic person we could find toward Copilot said the tool worked as exactly expected one time in ten. « When it guesses right, it feels like it’s reading my mind, » they said. GitHub calls it AI pair programming. But, right now, as it stands, Copilot is more like AI armchair programming, or backseat programming, than pair programming. This is not a team of equals. You, the human, are still ultimately and solely responsible, and that makes Copilot more autocomplete than robot programmer. A spokesperson for GitHub told us the Copilot team benchmarked their tool by making it complete functions in Python. The model generated correct code 43 per cent of the time on the first try, the PR rep said, and 57 per cent of the time when allowed 10 attempts. We’re assured it’s getting better. The spokesperson added that developers should use Copilot with the usual run of tests and security tools and apply their best judgment.
