Thanks to ChatGPT, he created an assistant for the iPhone that makes fun of Siri, Alexa and Google Assistant

A developer had the brilliant idea to use OpenAI bot as a home automation assistant. The result is amazing and should serve as a lesson to the assistants of Apple, Amazon or Google…

It’s just DIY, and yet it might just be a first look at the future of personal assistants. Mate Marschalko, a developer probably tired of the answers Siri gave him, had a brilliant idea: test ChatGPT to control his connected home with his voice.

In a video posted on Reddit and YouTube, he shows the -absolutely astonishing- result of his experiment, which sends Google, Apple and Amazon assistants back to the ropes.

If you regularly use Siri, Google Assistant or Alexa to control your light bulbs, radiators and other connected objects, you must have realized that you had to adapt to them. In other words, carry out very basic commands: “turn on the light in the living room”, “turn up the heating”, “turn off the TV”. Impossible to get them to understand more complex requests, so quickly they lose control.

Marschalko’s tool, built in an hour by combining the power of ChatGPT and Apple’s “Shortcuts” app, is a game-changer. And correctly interprets requests that are eminently more complicated than all GAFA assistants. An example ? This request, which would clearly have upset traditional voice assistants: ” My wife will come in 15 minutes. Turn on the light for her outside when she goes to park. The program understands her without the slightest concern, and the assistant proudly replies: The lights should come on the moment your guest arrives”. Or this one, who also benefits from the knowledge of ChatGPT: ” Adjust the heat in the bedroom to a temperature you think would help me sleep better “. And the assistant answered: The bedroom thermostat was set to 19 degrees. Enjoy your sleep! Ultimate sophistication, the wizard reacts differently every time, almost as if it had come to life.

A procedure… not that complicated

To connect his dozen lamps, thermostats, ventilation system and cameras to ChatGPT, he proceeded in two steps. First we had to “train” ChatGPT. Its long query (see below) looks like a small computer program… written in natural language.

Credit: Mate Marschalko // The query for ChatGPT looks like a natural language computer program

Thus, Marschalko required ChatGPT to respond to each request in the form of JSON, a famous data structuring format that can be easily understood by the shortcut application for iOS. He then described the types of requests needed, defined the structure of the JSON that ChatGPT should generate, and finally gave a precise description of his house and the connected objects that were in each room. That’s almost all. Ah yes, he asked too fine to imitate” the brains of the house, an intelligent AI, without revealing its true identity. »

Then he set up a new shortcut on iOS that looks like a supercharged version of that one we described to you a few days ago. Long and rather complex, this sequence of commands makes it possible, in summary, to create the interface, thanks to the data in JSON format, with its connected objects, via the Apple HomeKit platform.

It’s not (yet) right now

Ingenious and effective, Marschalko’s hack is not free of errors. First, there is the price. This solution actually requires access to ChatGPT through its API, and is therefore…payable, although OpenAI provides a (small) free credit. But it is quickly exhausted here, because the request is long. It takes about $1 every 70 requests, according to the programmer. Then there’s… the slowness. As such, the video can be confusing as it appears that ChatGPT is responding quickly. However, it has been mounted: there is an incompressible delay of a few seconds between the voice command and the robot’s response.

However, we can hope that Apple, Google or Amazon will also consider using generative AI to improve the relevance of their voice assistants. Which comes, with ChatGPT, to take a big hit from the old.


Mate Marshalko’s blog

Leave a Comment